Ekata Mitra
Author directory2026
Choosing the Right Language Mode at Inference Time for Multilingual Reliability
Ekata Mitra | Ameeta Agrawal
Findings of the Association for Computational Linguistics: EMNLP 2026
Ekata Mitra | Ameeta Agrawal
Findings of the Association for Computational Linguistics: EMNLP 2026
Multilingual large language models often struggle to reason in low- to mid-resource languages. Prior work has shown that translation can improve multilingual reasoning by helping models access stronger English-centric representations. This raises a central question: how much translation is needed for multilingual large language models to reason reliably, and when does additional translation instead trigger interference and overconfidence? Using LLaMA and Qwen models, we conduct extensive experiments that vary both text scope and language mode (target-only, English-only, and bilingual) to evaluate accuracy and reliability. Our results reveal a clear trade-off: English context often improves understanding and helps recover errors caused by non-English comprehension, yet adding redundant bilingual context can intensify interference. To address this trade-off, we propose Reliability-Aware Adaptive Inference (RAAI), a training-free test-time framework that (i) performs ECE-aware routing and prompt fusion and (ii) uses a mid-layer Risk Index (RI) to gate sequential reasoning, allocating extra compute only when it is likely to help while suppressing harmful bilingual redundancy. Across two model families, RAAI improves accuracy by 25 to 37.7 points in low-resource languages and reduces calibration error by about 3 to 6 points, with the largest gains appearing in the lowest-resource language tiers.
ThreadSumm: Summarization of Nested Discourse Threads Using Tree of Thoughts
Olubusayo Olabisi | Ekata Mitra | Ameeta Agrawal
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Olubusayo Olabisi | Ekata Mitra | Ameeta Agrawal
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Summarizing deeply nested discussion threads requires handling interleaved replies, quotes, and overlapping topics, which standard LLM summarizers struggle to capture reliably. We introduce ThreadSumm, a multi-stage LLM framework that treats thread summarization as a hierarchical reasoning problem over explicit aspect and content unit representations. Our method first performs content planning via LLM-based extraction of discourse aspects and Atomic Content Units, then applies sentence ordering to construct thread-aware sequences that surface multiple viewpoints rather than a single linear strand. On top of these interpretable units, ThreadSumm employs a Tree of Thoughts search that generates and scores multiple paragraph candidates, jointly optimizing coherence and coverage within a unified search space. With this multi-proposal and iterative refinement design, we show improved performance in generating logically structured summaries compared to existing baselines, while achieving higher aspect retention and opinion coverage in nested discussions.
2023
MEEP: Is this Engaging? Prompting Large Language Models for Dialogue Evaluation in Multilingual Settings
Amila Ferron | Amber Shore | Ekata Mitra | Ameeta Agrawal
Findings of the Association for Computational Linguistics: EMNLP 2023
Amila Ferron | Amber Shore | Ekata Mitra | Ameeta Agrawal
Findings of the Association for Computational Linguistics: EMNLP 2023
As dialogue systems become more popular, evaluation of their response quality gains importance. Engagingness highly correlates with overall quality and creates a sense of connection that gives human participants a more fulfilling experience. Although qualities like coherence and fluency are readily measured with well-worn automatic metrics, evaluating engagingness often relies on human assessment, which is a costly and time-consuming process. Existing automatic engagingness metrics evaluate the response without the conversation history, are designed for one dataset, or have limited correlation with human annotations. Furthermore, they have been tested exclusively on English conversations. Given that dialogue systems are increasingly available in languages beyond English, multilingual evaluation capabilities are essential. We propose that large language models (LLMs) may be used for evaluation of engagingness in dialogue through prompting, and ask how prompt constructs and translated prompts compare in a multilingual setting. We provide a prompt-design taxonomy for engagingness and find that using selected prompt elements with LLMs, including our comprehensive definition of engagingness, outperforms state-of-the-art methods on evaluation of engagingness in dialogue across multiple languages.