Interactive Plot Manipulation using Natural Language

Yihan Wang, Yutong Shao, Ndapa Nakashole


Abstract
We present an interactive Plotting Agent, a system that enables users to directly manipulate plots using natural language instructions within an interactive programming environment. The Plotting Agent maps language to plot updates. We formulate this problem as a slot-based task-oriented dialog problem, which we tackle with a sequence-to-sequence model. This plotting model while accurate in most cases, still makes errors, therefore, the system allows a feedback mode, wherein the user is presented with a top-k list of plots, among which the user can pick the desired one. From this kind of feedback, we can then, in principle, continuously learn and improve the system. Given that plotting is widely used across data-driven fields, we believe our demonstration will be of interest to both practitioners such as data scientists broadly defined, and researchers interested in natural language interfaces.
Anthology ID:
2021.naacl-demos.11
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstrations
Month:
June
Year:
2021
Address:
Online
Editors:
Avi Sil, Xi Victoria Lin
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
92–98
Language:
URL:
https://aclanthology.org/2021.naacl-demos.11
DOI:
10.18653/v1/2021.naacl-demos.11
Bibkey:
Cite (ACL):
Yihan Wang, Yutong Shao, and Ndapa Nakashole. 2021. Interactive Plot Manipulation using Natural Language. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstrations, pages 92–98, Online. Association for Computational Linguistics.
Cite (Informal):
Interactive Plot Manipulation using Natural Language (Wang et al., NAACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-demos.11.pdf
Video:
 https://aclanthology.org/2021.naacl-demos.11.mp4