Agnese Lombardi
Author directory2026
What Do You Mean? Exploring the Alleged Theory of Mind Abilities of Large Language Models
Agnese Lombardi | Alessandro Lenci
Transactions of the Association for Computational Linguistics, Volume 14
Agnese Lombardi | Alessandro Lenci
Transactions of the Association for Computational Linguistics, Volume 14
This study explores the capacity of Large Language Models (LLMs) to perform tasks requiring Theory of Mind (ToM), a critical component of pragmatic language understanding. Although previous work suggests that LLMs may exhibit emergent ToM abilities, this research examines whether such capabilities genuinely involve reasoning about beliefs or merely reflect the reliance on shallow statistical cues. Through a series of controlled experiments featuring indirect speech acts and verbal irony, we assess how belief contexts influence LLM interpretations. The results reveal that, although LLMs occasionally succeed in decoding communicative intentions, their performance is not attributable to human-like ToM reasoning. This work underscores the limitations of LLMs in simulating humanlike ToM and offers insight into their interpretive biases, contributing to a deeper understanding of their linguistic capabilities.1
Conversational Implicatures through the Lens of LLMs
Agnese Lombardi | Alessandro Lenci
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Agnese Lombardi | Alessandro Lenci
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Recent research has explored the capacity of Large Language Models (LLMs) to perform pragmatic reasoning and interpret complex pragmatic phenomena. However, such phenomena are inherently ambiguous, and even human evaluations are highly variable. Many existing studies directly compare human and model responses while assuming a single “correct” interpretation, thereby overlooking the natural variability that characterizes human pragmatic understanding. This raises two key issues: (1) the need for novel evaluation methods that account for interpretive variability and allow for meaningful comparison between humans and models, and (2) the potential limitations of current linguistic theories in capturing the richness of human pragmatic behavior. We propose that LLMs can serve not only as benchmarks for human-model alignment, but also as tools for investigating the nature of pragmatic phenomena and their relationship to linguistic theory. To this end, we developed a handcrafted dataset encompassing eight types of conversational implicatures. Our study addresses three main research questions: (1) Do LLMs process conversational implicatures differently from humans? (2) If so, how do these differences manifest? (3) What do these findings reveal about the cognitive capacities of LLMs and the explanatory adequacy of pragmatic theory?
ToM in LLM is not ToM, but a Pragmatic Effect
Agnese Lombardi | Alessandro Lenci
Proceedings of the 15th Workshop on Cognitive Modeling and Computational Linguistics
Agnese Lombardi | Alessandro Lenci
Proceedings of the 15th Workshop on Cognitive Modeling and Computational Linguistics
2025
Doing Things with Words: Rethinking Theory of Mind Simulation in Large Language Models
Agnese Lombardi | Alessandro Lenci
Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025)
Agnese Lombardi | Alessandro Lenci
Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025)