More Robust Schema-Guided Dialogue State Tracking via Tree-Based Paraphrase Ranking

Alexandru Coca, Bo-Hsiang Tseng, Weizhe Lin, Bill Byrne


Abstract
The schema-guided paradigm overcomes scalability issues inherent in building task-oriented dialogue (TOD) agents with static ontologies. Rather than operating on dialogue context alone, agents have access to hierarchical schemas containing task-relevant natural language descriptions. Fine-tuned language models excel at schema-guided dialogue state tracking (DST) but are sensitive to the writing style of the schemas. We explore methods for improving the robustness of DST models. We propose a framework for generating synthetic schemas which uses tree-based ranking to jointly optimise lexical diversity and semantic faithfulness. The robust generalisation of strong baselines is improved when augmenting their training data with prompts generated by our framework, as demonstrated by marked improvements in average Joint Goal Accuracy (JGA) and schema sensitivity (SS) on the SGD-X benchmark.
Anthology ID:
2023.findings-eacl.106
Volume:
Findings of the Association for Computational Linguistics: EACL 2023
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1443–1454
Language:
URL:
https://aclanthology.org/2023.findings-eacl.106
DOI:
10.18653/v1/2023.findings-eacl.106
Bibkey:
Cite (ACL):
Alexandru Coca, Bo-Hsiang Tseng, Weizhe Lin, and Bill Byrne. 2023. More Robust Schema-Guided Dialogue State Tracking via Tree-Based Paraphrase Ranking. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1443–1454, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
More Robust Schema-Guided Dialogue State Tracking via Tree-Based Paraphrase Ranking (Coca et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-eacl.106.pdf
Video:
 https://aclanthology.org/2023.findings-eacl.106.mp4