Weakly Supervised Semantic Parsing with Abstract Examples

Omer Goldman, Veronica Latcinnik, Ehud Nave, Amir Globerson, Jonathan Berant


Abstract
Training semantic parsers from weak supervision (denotations) rather than strong supervision (programs) complicates training in two ways. First, a large search space of potential programs needs to be explored at training time to find a correct program. Second, spurious programs that accidentally lead to a correct denotation add noise to training. In this work we propose that in closed worlds with clear semantic types, one can substantially alleviate these problems by utilizing an abstract representation, where tokens in both the language utterance and program are lifted to an abstract form. We show that these abstractions can be defined with a handful of lexical rules and that they result in sharing between different examples that alleviates the difficulties in training. To test our approach, we develop the first semantic parser for CNLVR, a challenging visual reasoning dataset, where the search space is large and overcoming spuriousness is critical, because denotations are either TRUE or FALSE, and thus random programs are likely to lead to a correct denotation. Our method substantially improves performance, and reaches 82.5% accuracy, a 14.7% absolute accuracy improvement compared to the best reported accuracy so far.
Anthology ID:
P18-1168
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1809–1819
Language:
URL:
https://aclanthology.org/P18-1168
DOI:
10.18653/v1/P18-1168
Bibkey:
Cite (ACL):
Omer Goldman, Veronica Latcinnik, Ehud Nave, Amir Globerson, and Jonathan Berant. 2018. Weakly Supervised Semantic Parsing with Abstract Examples. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1809–1819, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Weakly Supervised Semantic Parsing with Abstract Examples (Goldman et al., ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/P18-1168.pdf
Note:
 P18-1168.Notes.pdf
Presentation:
 P18-1168.Presentation.pdf
Video:
 https://aclanthology.org/P18-1168.mp4
Data
CLEVR