Whose Toxicity? Persona Conditioning and Model–Annotator Label Agreement by Gender in Indonesian Toxicity Classification

Salsabila Zahirah Pranida, Antonia Karamolegkou, Andrea Morales-Garzón, Daniel Hershcovich


Abstract
Toxicity judgments can depend on who is judging. We study whether persona prompting captures this variation by evaluating ten Indonesian-capable LLMs under four persona conditions on 7,524 IndoDiscourse posts, each labeled by at least one female and one male annotator, comprising 24,655 individual judgments. Persona cues substantially affect toxicity predictions, but their effects are model-specific, show no common direction, and are small between the two gender personas. Yet this steerability does not amount to perspective adoption: switching between gender personas does not selectively increase label agreement with the corresponding annotator group. Annotation-level mixed-effects models and annotator-level permutation tests likewise find no reliable female–male agreement gap. The apparent gap obtained from aggregated group labels instead follows from annotator label rates and aggregation choices.
Anthology ID:
2026.pandora-1.16
Volume:
Proceedings of the First Workshop on Pluralistic AI NLP: Diversity-aware, Sociotechnical, Responsible Alignment (PANDORA 2026)
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Shanshan Xu, Santosh T.Y.S.S., Kemal Kurniawan, Siyao Peng, Sarenne Wallbridge, Fajri Koto, Nouha Dziri, Barbara Plank
Venues:
PANDORA | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
279–300
Language:
URL:
https://aclanthology.org/2026.pandora-1.16/
DOI:
Bibkey:
Cite (ACL):
Salsabila Zahirah Pranida, Antonia Karamolegkou, Andrea Morales-Garzón, and Daniel Hershcovich. 2026. Whose Toxicity? Persona Conditioning and Model–Annotator Label Agreement by Gender in Indonesian Toxicity Classification. In Proceedings of the First Workshop on Pluralistic AI NLP: Diversity-aware, Sociotechnical, Responsible Alignment (PANDORA 2026), pages 279–300, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Whose Toxicity? Persona Conditioning and Model–Annotator Label Agreement by Gender in Indonesian Toxicity Classification (Pranida et al., PANDORA 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.pandora-1.16.pdf