Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking

Brendan King, Jeffrey Flanigan


Abstract
There has been significant interest in zero and few-shot learning for dialogue state tracking (DST) due to the high cost of collecting and annotating task-oriented dialogues. Recent work has demonstrated that in-context learning requires very little data and zero parameter updates, and even outperforms trained methods in the few-shot setting. We propose RefPyDST, which advances the state of the art with three advancements to in-context learning for DST.First, we formulate DST as a Python programming task, explicitly modeling language coreference as variable reference in Python. Second, since in-context learning depends highly on the context examples, we propose a method to retrieve a diverse set of relevant examples to improve performance. Finally, we introduce a novel re-weighting method during decoding that takes into account probabilities of competing surface forms, and produces a more accurate dialogue state prediction. We evaluate our approach using MultiWOZ and achieve state-of-the-art multi-domain joint-goal accuracy in zero and few-shot settings.
Anthology ID:
2023.findings-acl.344
Original:
2023.findings-acl.344v1
Version 2:
2023.findings-acl.344v2
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5570–5585
Language:
URL:
https://aclanthology.org/2023.findings-acl.344
DOI:
10.18653/v1/2023.findings-acl.344
Bibkey:
Cite (ACL):
Brendan King and Jeffrey Flanigan. 2023. Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5570–5585, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking (King & Flanigan, Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-acl.344.pdf
Video:
 https://aclanthology.org/2023.findings-acl.344.mp4