Rashmi Elavazhagan
Author directory2026
The Validity–Fidelity Gap: A Controlled Study of Representation Drift in Structured Extraction Under Redundant Contexts
Gokul Srinath Seetha Ram | Anirudh Vasudevan | Rashmi Elavazhagan
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Gokul Srinath Seetha Ram | Anirudh Vasudevan | Rashmi Elavazhagan
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Pipelines that use LLMs for structured extraction are commonly monitored by checking that each output parses as JSON and matches the expected keys. We show that this check can stay perfect while the extracted content degrades. We call the failure representation drift: the model still finds an entity but writes it in a different form than the intended one, for example a nested key-value object where a plain string is expected. The output is still valid JSON, yet the entity is scored both as missing (a false negative) and as an unexpected extra (a false positive). We study this with synthetic clinical notes repeated verbatim 1x-32x, which gives exact ground truth with prompt, schema and decoding held fixed. For LLaMA-3.1-8B-Instruct, micro-F1 falls from 0.499 to 0.259 while JSON validity stays 1.00. All 280 medication entities scored as missing in a 100-patient run are present in the output as nested objects, and a follow-up request with no schema recovers 96.7% of medications versus 40.0% under the schema at 16x. We introduce the Differential Redundancy Effect (DRE), the extra F1 drop under repeated content relative to an approximately length-matched unrelated-filler control, and use it as a diagnostic contrast rather than a causal estimate. The effect is scale-dependent: DRE is large at 8B and near zero at 70B and Qwen2.5-72B, where F1 does not degrade. Validity metrics stay at 1.00 in both regimes, so they carry no information about content fidelity. A smaller synthetic news experiment shows the same dissociation with a different schema. We also give a zero-ground-truth drift check and monitoring recommendations. All evidence is synthetic, and we make no claim about real clinical notes.
Consensus Drift: Semantic Salience Can Override Majority Evidence in LLM Aggregation
Gokul Srinath Seetha Ram | Rashmi Elavazhagan | Anirudh Vasudevan
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Gokul Srinath Seetha Ram | Rashmi Elavazhagan | Anirudh Vasudevan
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
We study how language models aggregate conflicting evidence in controlled multi-juror settings. As a null control, symbolic majority voting, where jurors provide only YES/NO votes, yields near-zero minority override across models, confirming this is not an artifact of the prompt family. Under semantic aggregation, where jurors instead provide short evidence statements paired with local verdicts, override becomes strength-dependent and position-sensitive: rates rise when a minority statement is operationalized as stronger than a weak majority, while a strong-majority/weak-minority control remains low-override. Authority cues applied to a minority statement can modulate this effect in some contexts, but do not universally dominate evidence strength. We validate these patterns with seed replications, cross-domain templates, phase-transition mapping across majority strength, minority strength, and position, and rationality-vs-salience controls. We further confirm the pattern on a larger open-weight model (Llama-3.3-70B-Instruct) and the GPT-4.1 flagship model: Llama-3.3-70B stays close to the smaller-model baseline, while GPT-4.1 shows a substantially larger override magnitude, indicating the effect’s size is model-dependent even as its qualitative shape persists. We emphasize that evidence “strength” here is a synthetic, operationalized manipulation rather than real-world ground truth, and that minority-following can be rational when minority evidence is genuinely decisive.
Search