Chong Peng
2024
Calibrating the Confidence of Large Language Models by Eliciting Fidelity
Mozhi Zhang
|
Mianqiu Huang
|
Rundong Shi
|
Linsen Guo
|
Chong Peng
|
Peng Yan
|
Yaqian Zhou
|
Xipeng Qiu
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Large language models optimized with techniques like RLHF have achieved good alignment in being helpful and harmless. However, post-alignment, these language models often exhibit overconfidence, where the expressed confidence does not accurately calibrate with their correctness rate. In this paper, we decompose the language model confidence into the Uncertainty about the question and the Fidelity to the answer generated by language models. Then, we propose a plug-and-play method, UF Calibration, to estimate the confidence of language models. Our method has shown good calibration performance by conducting experiments with 6 RLHF-LMs on four MCQA datasets. Moreover, we propose two novel metrics, IPR and CE, to evaluate the calibration of the model, and we have conducted a detailed discussion on Truly Well-Calibrated Confidence for large language models. Our method could serve as a strong baseline, and we hope that this work will provide some insights into the model confidence calibration.
Search
Co-authors
- Mozhi Zhang 1
- Mianqiu Huang 1
- Rundong Shi 1
- Linsen Guo 1
- Peng Yan 1
- show all...