On the Effects of Structural Modeling for Neural Semantic Parsing

Xiang Zhang, Shizhu He, Kang Liu, Jun Zhao


Abstract
Semantic parsing aims to map natural language sentences to predefined formal languages, such as logic forms and programming languages, as the semantic annotation. From the theoretic views of linguistic and programming language, structures play an important role in both languages, which had motivated semantic parsers since the task was proposed in the beginning. But in the neural era, semantic parsers treating both natural and formal language as sequences, such as Seq2Seq and LLMs, have got more attentions. On the other side, lots of neural progress have been made for grammar induction, which only focuses on natural languages. Although closely related in the sense of structural modeling, these techniques hadn’t been jointly analyzed on the semantic parsing testbeds. To gain the better understanding on structures for semantic parsing, we design a taxonomy of structural modeling methods, and evaluate some representative techniques on semantic parsing, including both compositional and i.i.d. generalizations. In addition to the previous opinion that structures will help in general, we find that (1) structures must be designed for the specific dataset and generalization level, and (2) what really matters is not the structure choice of either source or target side, but the choice combination of both sides. Based on the finding, we further propose a metric that can evaluate the structure choice, which we believe can boost the automation of grammar designs for specific datasets and domains.
Anthology ID:
2023.conll-1.4
Volume:
Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL)
Month:
December
Year:
2023
Address:
Singapore
Editors:
Jing Jiang, David Reitter, Shumin Deng
Venue:
CoNLL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
38–57
Language:
URL:
https://aclanthology.org/2023.conll-1.4
DOI:
10.18653/v1/2023.conll-1.4
Bibkey:
Cite (ACL):
Xiang Zhang, Shizhu He, Kang Liu, and Jun Zhao. 2023. On the Effects of Structural Modeling for Neural Semantic Parsing. In Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), pages 38–57, Singapore. Association for Computational Linguistics.
Cite (Informal):
On the Effects of Structural Modeling for Neural Semantic Parsing (Zhang et al., CoNLL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.conll-1.4.pdf
Software:
 2023.conll-1.4.Software.zip
Video:
 https://aclanthology.org/2023.conll-1.4.mp4