Resisting the Lure of the Skyline: Grounding Practices in Active Learning for Morphological Inflection

Saliha Muradoglu, Michael Ginn, Miikka Silfverberg, Mans Hulden


Abstract
Active learning (AL) aims to lower the demand of annotation by selecting informative unannotated samples for the model building. In this paper, we explore the importance of conscious experimental design in the language documentation and description setting, particularly the distribution of the unannotated sample pool. We focus on the task of morphological inflection using a Transformer model. We propose context motivated benchmarks: a baseline and skyline. The baseline describes the frequency weighted distribution encountered in natural speech. We simulate this using Wikipedia texts. The skyline defines the more common approach, uniform sampling from a large, balanced corpus (UniMorph, in our case), which often yields mixed results. We note the unrealistic nature of this unannotated pool. When these factors are considered, our results show a clear benefit to targeted sampling.
Anthology ID:
2024.acl-short.4
Volume:
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
47–55
Language:
URL:
https://aclanthology.org/2024.acl-short.4
DOI:
10.18653/v1/2024.acl-short.4
Bibkey:
Cite (ACL):
Saliha Muradoglu, Michael Ginn, Miikka Silfverberg, and Mans Hulden. 2024. Resisting the Lure of the Skyline: Grounding Practices in Active Learning for Morphological Inflection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 47–55, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Resisting the Lure of the Skyline: Grounding Practices in Active Learning for Morphological Inflection (Muradoglu et al., ACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.acl-short.4.pdf