Platt-Bin: Efficient Posterior Calibrated Training for NLP Classifiers

Rishabh Singh, Shirin Goshtasbpour


Abstract
Modern NLP classifiers are known to return uncalibrated estimations of class posteriors. Existing methods for posterior calibration rescale the predicted probabilities but often have an adverse impact on final classification accuracy, thus leading to poorer generalization. We propose an end-to-end trained calibrator, Platt-Binning, that directly optimizes the objective while minimizing the difference between the predicted and empirical posterior probabilities. Our method leverages the sample efficiency of Platt scaling and the verification guarantees of histogram binning, thus not only reducing the calibration error but also improving task performance. In contrast to existing calibrators, we perform this efficient calibration during training. Empirical evaluation of benchmark NLP classification tasks echoes the efficacy of our proposal.
Anthology ID:
2022.findings-acl.290
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3673–3684
Language:
URL:
https://aclanthology.org/2022.findings-acl.290
DOI:
10.18653/v1/2022.findings-acl.290
Bibkey:
Cite (ACL):
Rishabh Singh and Shirin Goshtasbpour. 2022. Platt-Bin: Efficient Posterior Calibrated Training for NLP Classifiers. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3673–3684, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Platt-Bin: Efficient Posterior Calibrated Training for NLP Classifiers (Singh & Goshtasbpour, Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.290.pdf
Software:
 2022.findings-acl.290.software.zip