ASSIST: Towards Label Noise-Robust Dialogue State Tracking

Fanghua Ye, Yue Feng, Emine Yilmaz


Abstract
The MultiWOZ 2.0 dataset has greatly boosted the research on dialogue state tracking (DST). However, substantial noise has been discovered in its state annotations. Such noise brings about huge challenges for training DST models robustly. Although several refined versions, including MultiWOZ 2.1-2.4, have been published recently, there are still lots of noisy labels, especially in the training set. Besides, it is costly to rectify all the problematic annotations. In this paper, instead of improving the annotation quality further, we propose a general framework, named ASSIST (lAbel noiSe-robuSt dIalogue State Tracking), to train DST models robustly from noisy labels. ASSIST first generates pseudo labels for each sample in the training set by using an auxiliary model trained on a small clean dataset, then puts the generated pseudo labels and vanilla noisy labels together to train the primary model. We show the validity of ASSIST theoretically. Experimental results also demonstrate that ASSIST improves the joint goal accuracy of DST by up to 28.16% on MultiWOZ 2.0 and 8.41% on MultiWOZ 2.4, compared to using only the vanilla noisy labels.
Anthology ID:
2022.findings-acl.214
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2719–2731
Language:
URL:
https://aclanthology.org/2022.findings-acl.214
DOI:
10.18653/v1/2022.findings-acl.214
Bibkey:
Cite (ACL):
Fanghua Ye, Yue Feng, and Emine Yilmaz. 2022. ASSIST: Towards Label Noise-Robust Dialogue State Tracking. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2719–2731, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
ASSIST: Towards Label Noise-Robust Dialogue State Tracking (Ye et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.214.pdf
Software:
 2022.findings-acl.214.software.zip
Video:
 https://aclanthology.org/2022.findings-acl.214.mp4
Code
 smartyfh/dst-assist
Data
MultiWOZSGD