Analogy Models for Neural Word Inflection

Ling Liu, Mans Hulden


Abstract
Analogy is assumed to be the cognitive mechanism speakers resort to in order to inflect an unknown form of a lexeme based on knowledge of other words in a language. In this process, an analogy is formed between word forms within an inflectional paradigm but also across paradigms. As neural network models for inflection are typically trained only on lemma-target form pairs, we propose three new ways to provide neural models with additional source forms to strengthen analogy-formation, and compare our methods to other approaches in the literature. We show that the proposed methods of providing a Transformer sequence-to-sequence model with additional analogy sources in the input are consistently effective, and improve upon recent state-of-the-art results on 46 languages, particularly in low-resource settings. We also propose a method to combine the analogy-motivated approach with data hallucination or augmentation. We find that the two approaches are complementary to each other and combining the two approaches is especially helpful when the training data is extremely limited.
Anthology ID:
2020.coling-main.257
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Donia Scott, Nuria Bel, Chengqing Zong
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
2861–2878
Language:
URL:
https://aclanthology.org/2020.coling-main.257
DOI:
10.18653/v1/2020.coling-main.257
Bibkey:
Cite (ACL):
Ling Liu and Mans Hulden. 2020. Analogy Models for Neural Word Inflection. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2861–2878, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
Analogy Models for Neural Word Inflection (Liu & Hulden, COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-main.257.pdf
Code
 linguistliu/analogy_for_inflection