FAST: Fast Annotation tool for SmarT devices

Shunyo Kawamoto, Yu Sawai, Kohei Wakimoto, Peinan Zhang


Abstract
Working with a wide range of annotators with the same attributes is crucial, as in real-world applications. Although such application cases often use crowd-sourcing mechanisms to gather a variety of annotators, most real-world users use mobile devices. In this paper, we propose “FAST,” an annotation tool for application tasks that focuses on the user experience of mobile devices, which has not yet been focused on thus far. We designed FAST as a web application for use on any device with a flexible interface that can be customized to fit various tasks. In our experiments, we conducted crowd-sourced annotation for a sentiment analysis task with several annotators and evaluated annotation metrics such as speed, quality, and ease of use from the tool’s logs and user surveys. Based on the results of our experiments, we conclude that our system can annotate faster than existing methods while maintaining the annotation quality.
Anthology ID:
2021.emnlp-demo.41
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Heike Adel, Shuming Shi
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
372–381
Language:
URL:
https://aclanthology.org/2021.emnlp-demo.41
DOI:
10.18653/v1/2021.emnlp-demo.41
Bibkey:
Cite (ACL):
Shunyo Kawamoto, Yu Sawai, Kohei Wakimoto, and Peinan Zhang. 2021. FAST: Fast Annotation tool for SmarT devices. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 372–381, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
FAST: Fast Annotation tool for SmarT devices (Kawamoto et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-demo.41.pdf
Software:
 2021.emnlp-demo.41.Software.zip
Video:
 https://aclanthology.org/2021.emnlp-demo.41.mp4
Code
 cyberagent/fast-annotation-tool