metaCAT: A Metadata-based Task-oriented Chatbot Annotation Tool

Ximing Liu, Wei Xue, Qi Su, Weiran Nie, Wei Peng


Abstract
Creating high-quality annotated dialogue corpora is challenging. It is essential to develop practical annotation tools to support humans in this time-consuming and error-prone task. We present metaCAT, which is an open-source web-based annotation tool designed specifically for developing task-oriented dialogue data. To the best of our knowledge, metaCAT is the first annotation tool that provides comprehensive metadata annotation coverage to the domain, intent, and span information. The data annotation quality is enhanced by a real-time annotation constraint-checking mechanism. An Automatic Speech Recognition (ASR) function is implemented to allow users to paraphrase and create more diversified annotated utterances. metaCAT is publicly available for the community.
Anthology ID:
2020.aacl-demo.4
Volume:
Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing: System Demonstrations
Month:
December
Year:
2020
Address:
Suzhou, China
Editors:
Derek Wong, Douwe Kiela
Venue:
AACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
20–25
Language:
URL:
https://aclanthology.org/2020.aacl-demo.4
DOI:
Bibkey:
Cite (ACL):
Ximing Liu, Wei Xue, Qi Su, Weiran Nie, and Wei Peng. 2020. metaCAT: A Metadata-based Task-oriented Chatbot Annotation Tool. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing: System Demonstrations, pages 20–25, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
metaCAT: A Metadata-based Task-oriented Chatbot Annotation Tool (Liu et al., AACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.aacl-demo.4.pdf
Code
 lexmen318/metaCAT
Data
MultiWOZ