Better Scores, Worse Grounding: Hidden Regressions after Fine-Tuning in Dialogue Fact Verification

Hyunkyung Park, Arkaitz Zubiaga


Abstract
In dialogue fact verification (DFV), responses often depend on prior turns for correct interpretation, yet systems are still judged mainly by aggregate benchmark scores. We study a hidden grounding regression: aggregate Macro-F1 improves after fine-tuning while previously correct, context-dependent pronoun cases become newly wrong and show stronger premise-side sensitivity than cases that remain correct. On three referent-annotated audit sets constructed from DialFact and FaithDial, we audit six encoder-only verifiers before and after matched source-specific fine-tuning through prediction-transition analysis and the Premise-Preference Score (PPS), a control-adjusted masking diagnostic. Fine-tuning improves Macro-F1 across the six-model/three-evaluation-set panel, yet newly regressed cases show stronger premise-side sensitivity than stable-correct cases under PPS in 17 of 18 evaluated comparisons, indicating that aggregate gains can conceal regressions on a controlled dialogue-grounding audit.
Anthology ID:
2026.sigdial-1.51
Volume:
Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Month:
August
Year:
2026
Address:
Atlanta, Georgia, USA
Editors:
Jinho D. Choi, Yun-Nung Chen, Kotaro Funakoshi, Ali Emami
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
720–737
Language:
URL:
https://aclanthology.org/2026.sigdial-1.51/
DOI:
Bibkey:
Cite (ACL):
Hyunkyung Park and Arkaitz Zubiaga. 2026. Better Scores, Worse Grounding: Hidden Regressions after Fine-Tuning in Dialogue Fact Verification. In Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 720–737, Atlanta, Georgia, USA. Association for Computational Linguistics.
Cite (Informal):
Better Scores, Worse Grounding: Hidden Regressions after Fine-Tuning in Dialogue Fact Verification (Park & Zubiaga, SIGDIAL 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.sigdial-1.51.pdf