Do Language Models Make Human-like Predictions about the Coreferents of Italian Anaphoric Zero Pronouns?

James A. Michaelov, Benjamin K. Bergen


Abstract
Some languages allow arguments to be omitted in certain contexts. Yet human language comprehenders reliably infer the intended referents of these zero pronouns, in part because they construct expectations about which referents are more likely. We ask whether Neural Language Models also extract the same expectations. We test whether 12 contemporary language models display expectations that reflect human behavior when exposed to sentences with zero pronouns from five behavioral experiments conducted in Italian by Carminati (2005). We find that three models - XGLM 2.9B, 4.5B, and 7.5B - capture the human behavior from all the experiments, with others successfully modeling some of the results. This result suggests that human expectations about coreference can be derived from exposure to language, and also indicates features of language models that allow them to better reflect human behavior.
Anthology ID:
2022.coling-1.1
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
1–14
Language:
URL:
https://aclanthology.org/2022.coling-1.1
DOI:
Bibkey:
Cite (ACL):
James A. Michaelov and Benjamin K. Bergen. 2022. Do Language Models Make Human-like Predictions about the Coreferents of Italian Anaphoric Zero Pronouns?. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1–14, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
Do Language Models Make Human-like Predictions about the Coreferents of Italian Anaphoric Zero Pronouns? (Michaelov & Bergen, COLING 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.coling-1.1.pdf
Code
 jmichaelov/italian-zero-anaphora-prediction