Subword Segmentation in LLMs: Looking at Inflection and Consistency

Subword Segmentation in LLMs: Looking at Inflection and Consistency Marion Di Marco author Alexander Fraser author 2024-11 text Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing Yaser Al-Onaizan editor Mohit Bansal editor Yun-Nung Chen editor Association for Computational Linguistics Miami, Florida, USA conference publication marco-fraser-2024-subword 10.18653/v1/2024.emnlp-main.672 https://aclanthology.org/2024.emnlp-main.672/ 2024-11 12050 12060