Sequence Reducible Holdout Loss for Language Model Pretraining

Sequence Reducible Holdout Loss for Language Model Pretraining Raghuveer Thirukovalluru author Nicholas Monath author Bhuwan Dhingra author Sam Wiseman author 2024-05 text Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) Nicoletta Calzolari editor Min-Yen Kan editor Veronique Hoste editor Alessandro Lenci editor Sakriani Sakti editor Nianwen Xue editor ELRA and ICCL Torino, Italia conference publication thirukovalluru-etal-2024-sequence https://aclanthology.org/2024.lrec-main.1281/ 2024-05 14705 14716