Crowdsourcing Complex Language Resources: Playing to Annotate Dependency Syntax

Bruno Guillaume, Karën Fort, Nicolas Lefebvre


Abstract
This article presents the results we obtained on a complex annotation task (that of dependency syntax) using a specifically designed Game with a Purpose, ZombiLingo. We show that with suitable mechanisms (decomposition of the task, training of the players and regular control of the annotation quality during the game), it is possible to obtain annotations whose quality is significantly higher than that obtainable with a parser, provided that enough players participate. The source code of the game and the resulting annotated corpora (for French) are freely available.
Anthology ID:
C16-1286
Volume:
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Yuji Matsumoto, Rashmi Prasad
Venue:
COLING
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
3041–3052
Language:
URL:
https://aclanthology.org/C16-1286
DOI:
Bibkey:
Cite (ACL):
Bruno Guillaume, Karën Fort, and Nicolas Lefebvre. 2016. Crowdsourcing Complex Language Resources: Playing to Annotate Dependency Syntax. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3041–3052, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
Crowdsourcing Complex Language Resources: Playing to Annotate Dependency Syntax (Guillaume et al., COLING 2016)
Copy Citation:
PDF:
https://aclanthology.org/C16-1286.pdf