Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomy

Vanessa Toborek, Florian Seiffarth, Sebastian Müller, Tamas Horvath


Abstract
Despite more than a decade of curriculum learning (CL) research in NLP, the field lacks a principled account of which difficulty function or scheduler to use for a given problem. To understand what has hindered progress towards this account, we propose a fine-grained taxonomy separating difficulty evaluation from training scheduling to enable systematic analysis of CL strategies. For difficulty evaluation, we distinguish attribution source and task dependence, revealing difficulty as a perspectival concept encoding different assumptions about what makes an instance hard to learn. For scheduling, we provide the first formalisation of CL schedulers in terms of expected training contribution, enabling comparison across implementations by introducing retention regimes and monotonicity properties. Applied in the first dedicated survey of CL in NLP, covering 107 strategies from 90 papers, our taxonomy reveals a systematic incomparability problem: prior works conflate distinct notions of difficulty and scheduling, often pursuing different objectives under the same CL label – hindering comparison and the accumulation of a coherent evidence base. Beyond diagnosis, the taxonomy supports the design, analysis, and comparison of CL strategies, and motivates evaluation practices that disentangle the sources of observed improvement.
Anthology ID:
2026.emnlp-main.1645
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
37384–37410
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1645/
DOI:
Bibkey:
Cite (ACL):
Vanessa Toborek, Florian Seiffarth, Sebastian Müller, and Tamas Horvath. 2026. Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomy. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 37384–37410, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomy (Toborek et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1645.pdf