Uncertainty Estimation and Reduction of Pre-trained Models for Text Regression

Yuxia Wang, Daniel Beck, Timothy Baldwin, Karin Verspoor


Abstract
State-of-the-art classification and regression models are often not well calibrated, and cannot reliably provide uncertainty estimates, limiting their utility in safety-critical applications such as clinical decision-making. While recent work has focused on calibration of classifiers, there is almost no work in NLP on calibration in a regression setting. In this paper, we quantify the calibration of pre- trained language models for text regression, both intrinsically and extrinsically. We further apply uncertainty estimates to augment training data in low-resource domains. Our experiments on three regression tasks in both self-training and active-learning settings show that uncertainty estimation can be used to increase overall performance and enhance model generalization.
Anthology ID:
2022.tacl-1.39
Volume:
Transactions of the Association for Computational Linguistics, Volume 10
Month:
Year:
2022
Address:
Cambridge, MA
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
680–696
Language:
URL:
https://aclanthology.org/2022.tacl-1.39
DOI:
10.1162/tacl_a_00483
Bibkey:
Cite (ACL):
Yuxia Wang, Daniel Beck, Timothy Baldwin, and Karin Verspoor. 2022. Uncertainty Estimation and Reduction of Pre-trained Models for Text Regression. Transactions of the Association for Computational Linguistics, 10:680–696.
Cite (Informal):
Uncertainty Estimation and Reduction of Pre-trained Models for Text Regression (Wang et al., TACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.tacl-1.39.pdf
Video:
 https://aclanthology.org/2022.tacl-1.39.mp4