From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood

Kelvin Guu, Panupong Pasupat, Evan Liu, Percy Liang


Abstract
Our goal is to learn a semantic parser that maps natural language utterances into executable programs when only indirect supervision is available: examples are labeled with the correct execution result, but not the program itself. Consequently, we must search the space of programs for those that output the correct result, while not being misled by spurious programs: incorrect programs that coincidentally output the correct result. We connect two common learning paradigms, reinforcement learning (RL) and maximum marginal likelihood (MML), and then present a new learning algorithm that combines the strengths of both. The new algorithm guards against spurious programs by combining the systematic search traditionally employed in MML with the randomized exploration of RL, and by updating parameters such that probability is spread more evenly across consistent programs. We apply our learning algorithm to a new neural semantic parser and show significant gains over existing state-of-the-art results on a recent context-dependent semantic parsing task.
Anthology ID:
P17-1097
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Editors:
Regina Barzilay, Min-Yen Kan
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1051–1062
Language:
URL:
https://aclanthology.org/P17-1097
DOI:
10.18653/v1/P17-1097
Bibkey:
Cite (ACL):
Kelvin Guu, Panupong Pasupat, Evan Liu, and Percy Liang. 2017. From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1051–1062, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood (Guu et al., ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-1097.pdf
Video:
 https://aclanthology.org/P17-1097.mp4
Code
 kelvinguu/lang2program +  additional community code