In LLMs, the source of information used to produce an answer is necessary to explain the generation process.This work considers the case where the answer can be extracted from the question’s context ( contextual knowledge) or from the model weights (parametric knowledge). While methods have been developed to uncover the grounding sources of LLMs in question answering, these approaches mainly rely on the text modality, whereas most tasks involve multiple modalities. This study proposes to uncover LLMs’ internal signals to determine whether they use parametric or contextual knowledge in a bimodal setting. To this end, we create a counterfactual dataset to evaluate whether the model relies on parametric or contextual knowledge on text and/or image-based questions. We leverage probing methods based on the internal attention and the different latent representations to determine the model’s source of information. We evaluate the performances of the probing methods, focusing on the different modalities. We show that models share common signals across the corpus and modalities. This study opens the way to develop controllable methods by uncovering common signals in LLMs to determine the knowledge used in generation.

We introduce the CAP (Confabulations from ACL Publications) dataset, a multilingual resource for studying hallucinations in large language models (LLMs) within scientific text generation. CAP focuses on the scientific domain, where hallucinations can distort factual knowledge, as they frequently do. In this domain, however, the presence of specialized terminology, statistical reasoning, and context-dependent interpretations further exacerbates these distortions, particularly given LLMs’ lack of true comprehension, limited contextual understanding, and bias toward surface-level generalization. CAP operates in a cross-lingual setting covering five high-resource languages (English, French, Hindi, Italian, and Spanish) and four low-resource languages (Bengali, Gujarati, Malayalam, and Telugu). The dataset comprises 900 curated scientific questions and over 7,000 LLM-generated answers from 16 publicly available models, provided as question–answer pairs along with token sequences and corresponding logits. Each instance is annotated with a binary label indicating the presence of a scientific hallucination, denoted as a factuality error, and a fluency label, capturing issues in the linguistic quality or naturalness of the text. CAP is publicly released to facilitate advanced research on hallucination detection, multilingual evaluation of LLMs, and the development of more reliable scientific NLP systems.

Evaluation is an essential process in the development and use of NLP systems. Benchmarks are widely used to calibrate systems, determine the efficiency of techniques, but also to advertise recently launched language models. Not only do benchmarks allow for evaluation and system comparisons, but they are also commonly utilized as training and validation corpora. In this paper, we aim at characterizing the "world-knowledge" that is put forth by benchmarks by closely inspecting their content. Through qualitative manual annotations (+15k instances) as well as quantitative automatic analyses, we uncover the presence of both explicit and implicit biases in MMLU, CommonSenseQA and MGSM, ranging from representational imbalance to bluntly sexist and racist instances. Our findings demonstrate that current NLP evaluation practices pose numerous quality issues, reinforce stereotypes and even contain explicitly harmful content. The current operationalization of benchmarks is questioned, and we provide recommendations and directions for further ethical evaluation. Warning: this paper contains content that may be offensive or upsetting.

This paper presents an overview of the SHROOM-CAP Shared Task, which focuses on detecting hallucinations and over-generation errors in cross-lingual analyses of scientific publications. SHROOM-CAP covers nine languages: five high-resource (English, French, Hindi, Italian, and Spanish) and four low-resource (Bengali, Gujarati, Malayalam, and Telugu). The task frames hallucination detection as a binary classification problem, where participants must predict whether a given text contains factual inaccuracies and fluency mistakes. We received 1,571 submissions from 5 participating teams during the test phase over the nine languages. In the paper, we present an analysis of the evaluated systems to assess their performance on the hallucination detection task across languages. Our findings reveal a disparity in system performance between high-resource and low-resource languages. Furthermore, we observe that factuality and fluency tend to be closely aligned in high-resource languages, whereas this correlation is less evident in low-resource languages. Overall, SHROOM-CAP underlines that hallucination detection remains a challenging open problem, particularly in low-resource and domain-specific settings.