Summarizing Behaviours: An Experiment on the Annotation of Call-Centre Conversations

Morena Danieli, Balamurali A R, Evgeny Stepanov, Benoit Favre, Frederic Bechet, Giuseppe Riccardi


Abstract
Annotating and predicting behavioural aspects in conversations is becoming critical in the conversational analytics industry. In this paper we look into inter-annotator agreement of agent behaviour dimensions on two call center corpora. We find that the task can be annotated consistently over time, but that subjectivity issues impacts the quality of the annotation. The reformulation of some of the annotated dimensions is suggested in order to improve agreement.
Anthology ID:
L16-1701
Volume:
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
Month:
May
Year:
2016
Address:
Portorož, Slovenia
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
4430–4433
Language:
URL:
https://aclanthology.org/L16-1701
DOI:
Bibkey:
Cite (ACL):
Morena Danieli, Balamurali A R, Evgeny Stepanov, Benoit Favre, Frederic Bechet, and Giuseppe Riccardi. 2016. Summarizing Behaviours: An Experiment on the Annotation of Call-Centre Conversations. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 4430–4433, Portorož, Slovenia. European Language Resources Association (ELRA).
Cite (Informal):
Summarizing Behaviours: An Experiment on the Annotation of Call-Centre Conversations (Danieli et al., LREC 2016)
Copy Citation:
PDF:
https://aclanthology.org/L16-1701.pdf