Mention detection with LLMs in pair-programming dialogue

Cecilia Domingo, Paul Piwek, Svetlana Stoyanchev, Michel Wermelinger


Abstract
We tackle the task of mention detection for pair-programming dialogue, a setting which adds several challenges to the task due to the characteristics of natural dialogue, the dynamic environment of the dialogue task, and the domain-specific vocabulary and structures. We compare recent variants of the Llama and GPT families and explore different prompt and context engineering approaches. While aspects like hesitations and references to read-out code and variable names made the task challenging, GPT 4.1 approximated human performance when we provided few-shot examples similar to the inference text and corrected formatting errors.
Anthology ID:
2025.crac-1.4
Volume:
Proceedings of the Eighth Workshop on Computational Models of Reference, Anaphora and Coreference
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Maciej Ogrodniczuk, Michal Novak, Massimo Poesio, Sameer Pradhan, Vincent Ng
Venue:
CRAC
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
42–54
Language:
URL:
https://aclanthology.org/2025.crac-1.4/
DOI:
Bibkey:
Cite (ACL):
Cecilia Domingo, Paul Piwek, Svetlana Stoyanchev, and Michel Wermelinger. 2025. Mention detection with LLMs in pair-programming dialogue. In Proceedings of the Eighth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 42–54, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Mention detection with LLMs in pair-programming dialogue (Domingo et al., CRAC 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.crac-1.4.pdf