This work revisits the task of training sequence tagging models with limited resources using transfer learning. We investigate several proposed approaches introduced in recent works and suggest a new loss that relies on sentence reconstruction from normalized embeddings. Specifically, our method demonstrates how by adding a decoding layer for sentence reconstruction, we can improve the performance of various baselines. We show improved results on the CoNLL02 NER and UD 1.2 POS datasets and demonstrate the power of the method for transfer learning with low-resources achieving 0.6 F1 score in Dutch using only one sample from it.
Tuning a Grammar Correction System for Increased Precision
Anoop Kunchukuttan | Sriram Chaudhury | Pushpak Bhattacharyya
Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task