Subword-Delimited Downsampling for Better Character-Level Translation

Lukas Edman, Antonio Toral, Gertjan van Noord


Abstract
Subword-level models have been the dominant paradigm in NLP. However, character-level models have the benefit of seeing each character individually, providing the model with more detailed information that ultimately could lead to better models. Recent works have shown character-level models to be competitive with subword models, but costly in terms of time and computation. Character-level models with a downsampling component alleviate this, but at the cost of quality, particularly for machine translation. This work analyzes the problems of previous downsampling methods and introduces a novel downsampling method which is informed by subwords.This new downsampling method not only outperforms existing downsampling methods, showing that downsampling characters can be done without sacrificing quality, but also leads to promising performance compared to subword models for translation.
Anthology ID:
2022.findings-emnlp.69
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2022
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
981–992
Language:
URL:
https://aclanthology.org/2022.findings-emnlp.69
DOI:
10.18653/v1/2022.findings-emnlp.69
Bibkey:
Cite (ACL):
Lukas Edman, Antonio Toral, and Gertjan van Noord. 2022. Subword-Delimited Downsampling for Better Character-Level Translation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 981–992, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Subword-Delimited Downsampling for Better Character-Level Translation (Edman et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-emnlp.69.pdf