Measuring Fine-Grained Semantic Equivalence with Abstract Meaning Representation

Shira Wein, Zhuxin Wang, Nathan Schneider


Abstract
Identifying semantically equivalent sentences is important for many NLP tasks. Current approaches to semantic equivalence take a loose, sentence-level approach to “equivalence,” despite evidence that fine-grained differences and implicit content have an effect on human understanding and system performance. In this work, we introduce a novel, more sensitive method of characterizing cross-lingual semantic equivalence that leverages Abstract Meaning Representation graph structures. We find that parsing sentences into AMRs and comparing the AMR graphs enables finer-grained equivalence measurement than comparing the sentences themselves. We demonstrate that when using gold or even automatically parsed AMR annotations, our solution is finer-grained than existing corpus filtering methods and more accurate at predicting strictly equivalent sentences than existing semantic similarity metrics.
Anthology ID:
2023.iwcs-1.16
Volume:
Proceedings of the 15th International Conference on Computational Semantics
Month:
June
Year:
2023
Address:
Nancy, France
Editors:
Maxime Amblard, Ellen Breitholtz
Venue:
IWCS
SIG:
SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
144–154
Language:
URL:
https://aclanthology.org/2023.iwcs-1.16
DOI:
Bibkey:
Cite (ACL):
Shira Wein, Zhuxin Wang, and Nathan Schneider. 2023. Measuring Fine-Grained Semantic Equivalence with Abstract Meaning Representation. In Proceedings of the 15th International Conference on Computational Semantics, pages 144–154, Nancy, France. Association for Computational Linguistics.
Cite (Informal):
Measuring Fine-Grained Semantic Equivalence with Abstract Meaning Representation (Wein et al., IWCS 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.iwcs-1.16.pdf