On the inconsistency of separable losses for structured prediction

Caio Corro


Abstract
In this paper, we prove that separable negative log-likelihood losses for structured prediction are not necessarily Bayes consistent, that is minimizing these losses may not result in a model that predicts the most probable structure in the data distribution for a given input. This fact opens the question of whether these losses are well-adapted for structured prediction and, if so, why.
Anthology ID:
2023.eacl-main.109
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1491–1498
Language:
URL:
https://aclanthology.org/2023.eacl-main.109
DOI:
10.18653/v1/2023.eacl-main.109
Bibkey:
Cite (ACL):
Caio Corro. 2023. On the inconsistency of separable losses for structured prediction. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1491–1498, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
On the inconsistency of separable losses for structured prediction (Corro, EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-main.109.pdf
Video:
 https://aclanthology.org/2023.eacl-main.109.mp4