Lusopol: How do LLMs Handle Politically Charged Questions? The Use Case of Portugal

Giuseppe Attanasio, Inês Vieira, Inês Calvo, Beatriz Canaverde, Ben Peters, Orfeas Menis Mastromichalakis, Miguel Faria, James Furtado, Iago Paulo, David Semedo, Joao Magalhaes, Andre Martins


Abstract
As large language models (LLMs) are adopted worldwide, evaluating their alignment across diverse cultural contexts is essential, especially when handling sensitive, politically charged requests. Yet what counts as “sensitive” is shaped by social, historical, and cultural factors, leaving model robustness across cultures poorly understood. We investigate this in the context of Portugal, a country whose layered history surfaces many culturally sensitive topics absent from existing benchmarks. We introduce Lusopol, a benchmark of 1,400 English and Portuguese prompts spanning seven thematic categories—from colonial history and the Estado Novo dictatorship to contemporary political issues—designed to stress-test models under deliberately controversial framings. Testing 16 state-of-the-art LLMs on Lusopol, we find substantial performance variance: frontier closed models and leading open-weight families set the pace, while European-focused open models lag behind. To close this gap, we synthesize a preference corpus of more than 7.5K samples. Targeted alignment training on this resource significantly reduces skewed responses and makes European open models competitive, demonstrating that culturally situated preference data is a practical path toward closing the alignment gap for underrepresented societies.
Anthology ID:
2026.findings-emnlp.2074
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
44149–44167
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2074/
DOI:
Bibkey:
Cite (ACL):
Giuseppe Attanasio, Inês Vieira, Inês Calvo, Beatriz Canaverde, Ben Peters, Orfeas Menis Mastromichalakis, Miguel Faria, James Furtado, Iago Paulo, David Semedo, Joao Magalhaes, and Andre Martins. 2026. Lusopol: How do LLMs Handle Politically Charged Questions? The Use Case of Portugal. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 44149–44167, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Lusopol: How do LLMs Handle Politically Charged Questions? The Use Case of Portugal (Attanasio et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2074.pdf