Parameter-Efficient Quality Estimation via Frozen Recursive Models

Umar Abubacar, Roman Bauer, Diptesh Kanojia


Abstract
Tiny Recursive Models (TRM) achieve strong results on reasoning tasks through iterative refinement of a shared network. We investigate whether these recursive mechanisms transfer to Quality Estimation (QE) for low-resource languages using a three-phase methodology. Experiments on 8 language pairs on a low-resource QE dataset reveal three findings. First, TRM’s recursive mechanisms do not transfer to QE. External iteration hurts performance, and internal recursion offers only narrow benefits. Next, representation quality dominates architectural choices, and lastly, frozen pretrained embeddings match fine-tuned performance while reducing trainable parameters by 37× (7M vs 262M). TRM-QE with frozen XLM-R embeddings achieves a Spearman’s correlation of 0.370, matching fine-tuned variants (0.369) and outperforming an equivalent-depth standard transformer (0.336). On Hindi and Tamil, frozen TRM-QE outperforms MonoTransQuest (560M parameters) with 80× fewer trainable parameters, suggesting that weight sharing combined with frozen embeddings enables parameter efficiency for QE.
Anthology ID:
2026.loreslm-1.52
Volume:
Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026)
Month:
March
Year:
2026
Address:
Rabat, Morocco
Editors:
Hansi Hettiarachchi, Tharindu Ranasinghe, Alistair Plum, Paul Rayson, Ruslan Mitkov, Mohamed Gaber, Damith Premasiri, Fiona Anting Tan, Lasitha Uyangodage
Venue:
LoResLM
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
609–614
Language:
URL:
https://aclanthology.org/2026.loreslm-1.52/
DOI:
Bibkey:
Cite (ACL):
Umar Abubacar, Roman Bauer, and Diptesh Kanojia. 2026. Parameter-Efficient Quality Estimation via Frozen Recursive Models. In Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026), pages 609–614, Rabat, Morocco. Association for Computational Linguistics.
Cite (Informal):
Parameter-Efficient Quality Estimation via Frozen Recursive Models (Abubacar et al., LoResLM 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.loreslm-1.52.pdf