Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text — A Hindi–English Case Study

Debajyoti Mazumder, Divyansh Pathak, Prashant Kodali, Jasabanta Patro


Abstract
Multilingual encoder-based language models are widely used for code-mixed analysis, yet their internal representations of code-mixed inputs—and their relationship to the constituent languages—remain poorly understood. Using Hindi–English as a case study, we construct a unified trilingual corpus of parallel English, Hindi (Devanagari), and Romanized code-mixed sentences. We then probe cross-lingual representation alignment in standard multilingual encoders and their code-mix-adapted variants using CKA, token-level saliency, and entropy-based uncertainty analysis. We find that while standard models align English and Hindi well, code-mixed inputs remain loosely connected to either language — and that continued pre-training on code-mixed data improves English–code-mixed alignment at the cost of English–Hindi alignment. Interpretability analyses further reveal a clear asymmetry: models process code-mixed text through an English-dominant semantic subspace, while native-script Hindi provides complementary signals that reduce representational uncertainty. Motivated by these findings, we introduce a trilingual post-training alignment objective that brings code-mixed representations closer to both constituent languages simultaneously, yielding more balanced cross-lingual alignment and downstream gains on sentiment analysis and hate speech detection — showing that grounding code-mixed representations in their constituent languages meaningfully helps cross-lingual understanding. Code is available at: https://github.com/debajyotimaz/tri_align_EMNLP_2026.
Anthology ID:
2026.findings-emnlp.1748
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
37390–37415
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1748/
DOI:
Bibkey:
Cite (ACL):
Debajyoti Mazumder, Divyansh Pathak, Prashant Kodali, and Jasabanta Patro. 2026. Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text — A Hindi–English Case Study. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 37390–37415, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text — A Hindi–English Case Study (Mazumder et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1748.pdf