Yihan Yu
Author directory2026
What AI Gets to See: Auditing the Loss of Multilingual Community Knowledge in Machine-Facing Metadata on Wikimedia Commons
Yihan Yu | Aylin Caliskan
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yihan Yu | Aylin Caliskan
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
AI systems increasingly inherit what public knowledge infrastructures make machine-visible. Yet multilingual NLP still tends to frame data responsibility around new corpora, annotations, and benchmarks, even though public knowledge communities have already produced rich multilingual knowledge that may be lost when AI-facing infrastructures compress it into narrower metadata layers. We study this problem on Wikimedia Commons, where multilingual captions and descriptions coexist with the English-dominant category system and Wikidata-linked Structured Data on Commons. Using 2,500 featured images across five domains, we introduce a representational recovery framework that audits lexical, semantic, and retrieval-based recovery from current machine-facing metadata layers.
Across these tests, we find a consistent asymmetry. After removing the exact query text from the gold image’s free-text document, Commons categories plus Wikidata entity labels outperform the remaining free text for English retrieval. For non-English retrieval, however, the remaining multilingual captions and descriptions outperform categories plus Wikidata entity labels rendered in the query language. Entity-label rendering improves non-English retrieval, but does not fully recover the meaning already present in contributor-written multilingual text. These results show that multilingual NLP should not only collect new data, but also audit, identify, and repair recovery gaps in public knowledge infrastructures as they increasingly feed large-scale AI training pipelines.
Across these tests, we find a consistent asymmetry. After removing the exact query text from the gold image’s free-text document, Commons categories plus Wikidata entity labels outperform the remaining free text for English retrieval. For non-English retrieval, however, the remaining multilingual captions and descriptions outperform categories plus Wikidata entity labels rendered in the query language. Entity-label rendering improves non-English retrieval, but does not fully recover the meaning already present in contributor-written multilingual text. These results show that multilingual NLP should not only collect new data, but also audit, identify, and repair recovery gaps in public knowledge infrastructures as they increasingly feed large-scale AI training pipelines.