Linking language model predictions to human behaviour on scalar implicatures

Yulia Zinova, David Arps, Katharina Spalek, Jacopo Romoli


Abstract
We explore the behaviour of language models on adjectival scales in connection with negation when prompted with material used in human experiments. We propose several metrics extracted from the model predictions and analyze those metrics in relation to human data as well as use them to propose new items to be tested in human experiments.
Anthology ID:
2025.neusymbridge-1.10
Volume:
Proceedings of Bridging Neurons and Symbols for Natural Language Processing and Knowledge Graphs Reasoning @ COLING 2025
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Kang Liu, Yangqiu Song, Zhen Han, Rafet Sifa, Shizhu He, Yunfei Long
Venues:
NeusymBridge | WS
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
97–106
Language:
URL:
https://aclanthology.org/2025.neusymbridge-1.10/
DOI:
Bibkey:
Cite (ACL):
Yulia Zinova, David Arps, Katharina Spalek, and Jacopo Romoli. 2025. Linking language model predictions to human behaviour on scalar implicatures. In Proceedings of Bridging Neurons and Symbols for Natural Language Processing and Knowledge Graphs Reasoning @ COLING 2025, pages 97–106, Abu Dhabi, UAE. ELRA and ICCL.
Cite (Informal):
Linking language model predictions to human behaviour on scalar implicatures (Zinova et al., NeusymBridge 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.neusymbridge-1.10.pdf