Weakly Supervised Word Segmentation for Computational Language Documentation

Shu Okabe, Laurent Besacier, François Yvon


Abstract
Word and morpheme segmentation are fundamental steps of language documentation as they allow to discover lexical units in a language for which the lexicon is unknown. However, in most language documentation scenarios, linguists do not start from a blank page: they may already have a pre-existing dictionary or have initiated manual segmentation of a small part of their data. This paper studies how such a weak supervision can be taken advantage of in Bayesian non-parametric models of segmentation. Our experiments on two very low resource languages (Mboshi and Japhug), whose documentation is still in progress, show that weak supervision can be beneficial to the segmentation quality. In addition, we investigate an incremental learning scenario where manual segmentations are provided in a sequential manner. This work opens the way for interactive annotation tools for documentary linguists.
Anthology ID:
2022.acl-long.510
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7385–7398
Language:
URL:
https://aclanthology.org/2022.acl-long.510
DOI:
10.18653/v1/2022.acl-long.510
Bibkey:
Cite (ACL):
Shu Okabe, Laurent Besacier, and François Yvon. 2022. Weakly Supervised Word Segmentation for Computational Language Documentation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7385–7398, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Weakly Supervised Word Segmentation for Computational Language Documentation (Okabe et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.510.pdf
Code
 shuokabe/pyseg