GPTs Are Multilingual Annotators for Sequence Generation Tasks

Juhwan Choi, Eunju Lee, Kyohoon Jin, YoungBin Kim


Abstract
Data annotation is an essential step for constructing new datasets. However, the conventional approach of data annotation through crowdsourcing is both time-consuming and expensive. In addition, the complexity of this process increases when dealing with low-resource languages owing to the difference in the language pool of crowdworkers. To address these issues, this study proposes an autonomous annotation method by utilizing large language models, which have been recently demonstrated to exhibit remarkable performance. Through our experiments, we demonstrate that the proposed method is not just cost-efficient but also applicable for low-resource language annotation. Additionally, we constructed an image captioning dataset using our approach and are committed to open this dataset for future study. We have opened our source code for further study and reproducibility.
Anthology ID:
2024.findings-eacl.2
Volume:
Findings of the Association for Computational Linguistics: EACL 2024
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
17–40
Language:
URL:
https://aclanthology.org/2024.findings-eacl.2
DOI:
Bibkey:
Cite (ACL):
Juhwan Choi, Eunju Lee, Kyohoon Jin, and YoungBin Kim. 2024. GPTs Are Multilingual Annotators for Sequence Generation Tasks. In Findings of the Association for Computational Linguistics: EACL 2024, pages 17–40, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
GPTs Are Multilingual Annotators for Sequence Generation Tasks (Choi et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-eacl.2.pdf
Software:
 2024.findings-eacl.2.software.zip
Note:
 2024.findings-eacl.2.note.zip
Video:
 https://aclanthology.org/2024.findings-eacl.2.mp4