Many Bangla words are at once personal names and culturally loaded common nouns: মায়া (Maya) is both a girl’s name and a word for affectionate compassion. Choosing the right reading demands cultural knowledge that is scarce in the pretraining data of modern language models. We introduce Bangla Culturally Entangled Homograph (BanglaCEH) disambiguation and build a Bangla benchmark of 1,516 expert-verified sentences and 3,032 labelled occurrences in which one word appears twice with two distinct readings, each labelled with a culturally grounded category and an explanation of the reasoning behind it. Across open- and closed-source models, we find a systematic dominant-meaning bias, models default to the common-noun sense and overlook the name. A Bangla-specific model fails under every prompting regime we test, showing that language-specific pretraining alone does not confer cultural grounding. We further show that contrastive chain-of-thought prompting can sharply reduce this bias without training, and that distilling cultural explanations teaches small (1–3B) models to reason toward the correct reading rather than memorise labels, cutting dominant-meaning bias from as high as 100% to under 5% and turning the failed Bangla-specific model into our strongest system.

The rise of Large Language Models (LLMs) has redefined Machine Translation (MT), enabling context-aware and fluent translations across hundreds of languages and textual domains. Despite their remarkable capabilities, LLMs often exhibit uneven performance across language families and specialized domains. Moreover, recent evidence reveals that these models can encode and amplify different biases present in their training data, posing serious concerns for fairness, especially in low-resource languages. To address these gaps, we introduce Translation Tangles, a unified framework and dataset for evaluating the translation quality and fairness of open-source LLMs. Our approach benchmarks 24 bidirectional language pairs across multiple domains using different metrics. We further propose a hybrid bias detection pipeline that integrates rule-based heuristics, semantic similarity filtering, and LLM-based validation. We also introduce a high-quality, bias-annotated dataset based on human evaluations of 1,439 translation-reference pairs. The code and dataset are accessible on GitHub: https://github.com/faiyazabdullah/TranslationTangles

The rapid growth of pre-trained models has made selecting suitable models for downstream tasks challenging. Exhaustively fine-tuning or evaluating every candidate model is computa- tionally expensive, while existing proxy-based metrics such as LEEP, NCE, and LogME pro- duce one-shot static rankings that do not use feedback from earlier evaluations. We propose Reinforcement Learning–based Model Rank- ing (RL-MR), a budget-aware framework that reformulates pretrained model selection as a sequential decision-making problem. RL-MR learns a ranking policy that uses proxy-based transferability signals as rewards to guide the order in which candidate models are evaluated. In controlled experiments on SST-2, Yelp Po- larity, Amazon Polarity, and DynaSent, RL- MR consistently improves ranking correlation over static proxy baselines, achieving up to Kendall’s τ = 0.8695 while maintaining Top-1 selection accuracy. These results suggest that sequential proxy feedback can provide a use- ful alternative to one-shot model ranking under limited evaluation budgets.