What AI Gets to See: Auditing the Loss of Multilingual Community Knowledge in Machine-Facing Metadata on Wikimedia Commons

Yihan Yu, Aylin Caliskan


Abstract
AI systems increasingly inherit what public knowledge infrastructures make machine-visible. Yet multilingual NLP still tends to frame data responsibility around new corpora, annotations, and benchmarks, even though public knowledge communities have already produced rich multilingual knowledge that may be lost when AI-facing infrastructures compress it into narrower metadata layers. We study this problem on Wikimedia Commons, where multilingual captions and descriptions coexist with the English-dominant category system and Wikidata-linked Structured Data on Commons. Using 2,500 featured images across five domains, we introduce a representational recovery framework that audits lexical, semantic, and retrieval-based recovery from current machine-facing metadata layers.

Across these tests, we find a consistent asymmetry. After removing the exact query text from the gold image’s free-text document, Commons categories plus Wikidata entity labels outperform the remaining free text for English retrieval. For non-English retrieval, however, the remaining multilingual captions and descriptions outperform categories plus Wikidata entity labels rendered in the query language. Entity-label rendering improves non-English retrieval, but does not fully recover the meaning already present in contributor-written multilingual text. These results show that multilingual NLP should not only collect new data, but also audit, identify, and repair recovery gaps in public knowledge infrastructures as they increasingly feed large-scale AI training pipelines.
Anthology ID:
2026.emnlp-main.2430
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
55176–55188
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2430/
DOI:
Bibkey:
Cite (ACL):
Yihan Yu and Aylin Caliskan. 2026. What AI Gets to See: Auditing the Loss of Multilingual Community Knowledge in Machine-Facing Metadata on Wikimedia Commons. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 55176–55188, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
What AI Gets to See: Auditing the Loss of Multilingual Community Knowledge in Machine-Facing Metadata on Wikimedia Commons (Yu & Caliskan, EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2430.pdf