Ahmer Tabassum
Author directory2026
The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering
Debopriyo Banerjee | Owais Aijaz | Ahmer Tabassum | Zhuohan Xie | Preslav Nakov | Veselin Stoyanov
Findings of the Association for Computational Linguistics: EMNLP 2026
Debopriyo Banerjee | Owais Aijaz | Ahmer Tabassum | Zhuohan Xie | Preslav Nakov | Veselin Stoyanov
Findings of the Association for Computational Linguistics: EMNLP 2026
Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunked pipeline has structural blind spots: it cannot anticipate which passages a question requires and discards document-level context that resolves cross-passage references. We introduce evidence-based generation (EBG), requiring models to return an answer with line-level citations, and a joint Ans+Evid metric that scores correctness and grounding simultaneously. We reformulate two corpora into evidence-grounded multi-document QA benchmarks—MultiHop-RAG and QASPER-MP, a multi-paper adaptation of QASPER with realistic short paper cues—and pair them with a controlled synthetic two-hop haystack and an evidence-augmented NoLiMa spanning 10K–1M characters. Across eight frontier proprietary and three open-weight LLMs, EBG isolates two RAG limitations that long-context models avoid: similarity retrieval surfaces any relevant chunk far more reliably than all required chunks for multi-fact queries, and chunking destroys document meta-information needed to resolve identity-bearing queries. EBG also exposes grounding failures invisible to answer-only evaluation, including incomplete citations in two-needle settings and a widening gap between Ans and Ans+Evid when context contradicts parametric knowledge. Data and code are available at https://github.com/debobanerjee/LC-EBG.
UrduMMLU: A Massive Multitask Benchmark for Urdu Language Understanding
Ahmer Tabassum | Sarfraz Ahmad | Hasan Iqbal | Owais Aijaz | Momina Ahsan | Preslav Nakov
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Ahmer Tabassum | Sarfraz Ahmad | Hasan Iqbal | Owais Aijaz | Momina Ahsan | Preslav Nakov
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Meaningful multilingual evaluation must test models in the target language and educational context. Urdu, spoken by more than 230 million people, lacks a broad MMLU-style benchmark built from native educational sources. We introduce UrduMMLU, a benchmark of 26,389 Urdu MCQs across 26 subjects and five domains, collected from native Urdu MCQ banks and public examination PDFs. Unlike translation-based benchmarks, UrduMMLU combines academic subjects with content specific to Urdu and regional education. We label the exam-derived portion through dual human annotation with strict consensus filtering. We evaluate 30 LLMs under English and Urdu prompts, yielding 60 zero-shot evaluations, and further evaluate four open-source LLMs under multiple few-shot settings across both prompt languages. Gemini-3.5-Flash performs best, reaching 90.23% and 90.45% accuracy, while no other model exceeds 85%. The strongest open-source model trails by 7.78 and 9.12 points, and many models lose 25 to 40 points on Urdu-centered Humanities subjects compared with STEM. Few-shot prompting yields only modest gains. Results on UrduMMLU show that current LLMs have uneven Urdu knowledge, particularly for content grounded in the regional context.