Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunked pipeline has structural blind spots: it cannot anticipate which passages a question requires and discards document-level context that resolves cross-passage references. We introduce evidence-based generation (EBG), requiring models to return an answer with line-level citations, and a joint Ans+Evid metric that scores correctness and grounding simultaneously. We reformulate two corpora into evidence-grounded multi-document QA benchmarks—MultiHop-RAG and QASPER-MP, a multi-paper adaptation of QASPER with realistic short paper cues—and pair them with a controlled synthetic two-hop haystack and an evidence-augmented NoLiMa spanning 10K–1M characters. Across eight frontier proprietary and three open-weight LLMs, EBG isolates two RAG limitations that long-context models avoid: similarity retrieval surfaces any relevant chunk far more reliably than all required chunks for multi-fact queries, and chunking destroys document meta-information needed to resolve identity-bearing queries. EBG also exposes grounding failures invisible to answer-only evaluation, including incomplete citations in two-needle settings and a widening gap between Ans and Ans+Evid when context contradicts parametric knowledge. Data and code are available at https://github.com/debobanerjee/LC-EBG.

Meaningful multilingual evaluation must test models in the target language and educational context. Urdu, spoken by more than 230 million people, lacks a broad MMLU-style benchmark built from native educational sources. We introduce UrduMMLU, a benchmark of 26,389 Urdu MCQs across 26 subjects and five domains, collected from native Urdu MCQ banks and public examination PDFs. Unlike translation-based benchmarks, UrduMMLU combines academic subjects with content specific to Urdu and regional education. We label the exam-derived portion through dual human annotation with strict consensus filtering. We evaluate 30 LLMs under English and Urdu prompts, yielding 60 zero-shot evaluations, and further evaluate four open-source LLMs under multiple few-shot settings across both prompt languages. Gemini-3.5-Flash performs best, reaching 90.23% and 90.45% accuracy, while no other model exceeds 85%. The strongest open-source model trails by 7.78 and 9.12 points, and many models lose 25 to 40 points on Urdu-centered Humanities subjects compared with STEM. Few-shot prompting yields only modest gains. Results on UrduMMLU show that current LLMs have uneven Urdu knowledge, particularly for content grounded in the regional context.

Multimodal retrieval-augmented generation (RAG) is usually evaluated with clean evidence, although deployed systems may retrieve topically relevant but unreliable text or images. We introduce QIMG-7, a controlled post-retrieval stress test for multi-sentence factual QA comprising 110 questions from four datasets, seven visual-evidence pollution families, and 16 paired clean/polluted regimes per question (1,760 rows per method). Across four generator/gate stacks, Full-MM is fragile under polluted text: in the main gpt-4o-mini stack, support decreases from 0.908 with clean text to 0.490 with polluted text. We propose source-aware trust resolution (SATR), a training-free family that compares Parametric, Text-only, and Full-MM candidate answers and selectively chooses a branch or falls back. Field-Selector obtains the highest numerical balanced score, 0.816, improving over Full-MM by 11.7 points and the Cascaded Router by 2.2 points. TextTrust, an image-blind text-reliability baseline, reaches 0.800; Field-Selector’s 1.5-point balanced advantage over it is not statistically resolved (95% CI [-0.006, 0.036]). In this text-first setting, the results support selective trust rather than unconditional multimodal fusion. Artifacts are available at https://github.com/mbzuai-nlp/Trust_Before_Fusion.

SemEval Task 6: Promise Eval, was designed to evaluate a company’s adherence to its ESG commitments. Using Natural Language Processing (NLP) and Deep Learning techniques, the task involves analyzing ESG reports to identify, classify, and verify corporate promises. The verification process follows a structured pipeline with four subtasks: Promise Classification, Evidence Verification, Evidence Classification, and Timeline Verification. These subtasks ensure that identified promises are well-defined, supported by credible evidence, and time-bound.For model implementation, BERT was initially used for most of the classification tasks but was later replaced with DeBERTa, which improved performance due to its superior contextual understanding. To enhance model generalization, contrastive learning was applied alongside standard classification loss, helping the model differentiate between positive and negative examples. Oversampling techniques were used to address class imbalance issues, particularly for the Misleading evidence category. For timeline verification, BART was chosen initially but then shifted to DeBERTa again, as it better captures sequential dependencies in text.The dataset consists of ESG reports containing labeled promise statements, evidence snippets, and timeline information. The data was preprocessed by tokenizing text, handling imbalanced classes through oversampling, and incorporating domain-specific embeddings to improve understanding.By implementing these techniques, the research aims to provide a transparent and accountable framework for assessing corporate promises, ensuring that companies are held accountable for their ESG commitments.