Enhancing and Evaluating the Grammatical Framework Approach to Logic-to-Text Generation

Eduardo Calò, Elze van der Werf, Albert Gatt, Kees van Deemter


Abstract
Logic-to-text generation is an important yet underrepresented area of natural language generation (NLG). In particular, most previous works on this topic lack sound evaluation. We address this limitation by building and evaluating a system that generates high-quality English text given a first-order logic (FOL) formula as input. We start by analyzing the performance of Ranta (2011)’s system. Based on this analysis, we develop an extended version of the system, which we name LoLa, that performs formula simplification based on logical equivalences and syntactic transformations. We carry out an extensive evaluation of LoLa using standard automatic metrics and human evaluation. We compare the results against a baseline and Ranta (2011)’s system. The results show that LoLa outperforms the other two systems in most aspects.
Anthology ID:
2022.gem-1.13
Volume:
Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM)
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates (Hybrid)
Editors:
Antoine Bosselut, Khyathi Chandu, Kaustubh Dhole, Varun Gangal, Sebastian Gehrmann, Yacine Jernite, Jekaterina Novikova, Laura Perez-Beltrachini
Venue:
GEM
SIG:
SIGGEN
Publisher:
Association for Computational Linguistics
Note:
Pages:
148–171
Language:
URL:
https://aclanthology.org/2022.gem-1.13
DOI:
10.18653/v1/2022.gem-1.13
Bibkey:
Cite (ACL):
Eduardo Calò, Elze van der Werf, Albert Gatt, and Kees van Deemter. 2022. Enhancing and Evaluating the Grammatical Framework Approach to Logic-to-Text Generation. In Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), pages 148–171, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Cite (Informal):
Enhancing and Evaluating the Grammatical Framework Approach to Logic-to-Text Generation (Calò et al., GEM 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.gem-1.13.pdf
Video:
 https://aclanthology.org/2022.gem-1.13.mp4