Ahmed Asaad

Author directory

2026

Large Language Models (LLMs) use persistent user context such as memory, profiles, and role prompts. This enables personalization, but can destabilize evidence-based judgment: the same evidence may yield different conclusions under different user contexts. Finance offers a high-stakes long-document setting where evidence reading is central. We test this on 3,575 SEC filings across twelve models. By comparing persona-conditioned retrieval, neutral retrieval, and memory-framed context, we find that bias comes from how models interpret identical evidence under role-style context, not from which evidence they retrieve. As mitigation, user-memory framing of the same mindset and dual-output scoring reduce the effect, with model-specific effectiveness across models.
Large language models are increasingly used to answer culturally grounded questions across languages, yet it remains unclear whether local cultural knowledge is better accessed through English or the local language. Existing evaluations face two key limitations: many rely on parallel template-based questions that may not reflect how cultural knowledge naturally appears, and raw accuracy conflates general language proficiency with language-conditioned knowledge access. We address these issues with a controlled framework built on real-world cultural questions collected from regional benchmarks and local sources. By crossing question type, culture-agnostic versus culture-specific, with query language, English versus local language, and estimating ability using a shared 1PL item response theory model, we separate proficiency from localized knowledge access. Across 13 locales and roughly 80 models, we find a consistent English advantage on culture-agnostic questions, indicating stronger English proficiency. However, after accounting for this proficiency gap, local languages show a positive knowledge-access advantage in nearly all locale–model settings. This advantage is often masked in raw accuracy but becomes more visible for frontier, regionally aligned, or language-adapted models. Our results suggest that weaker local-language performance does not necessarily imply weaker cultural knowledge; rather, local cultural knowledge may be more accessible through the local language but hidden by limited language proficiency.