Improving Factual Accuracy of Neural Table-to-Text Output by Addressing Input Problems in ToTTo

Barkavi Sundararajan, Yaji Sripada, Ehud Reiter


Abstract
Neural Table-to-Text models tend to hallucinate, producing texts that contain factual errors. We investigate whether such errors in the output can be traced back to problems with the input. We manually annotated 1,837 texts generated by multiple models in the politics domain of the ToTTo dataset. We identify the input problems that are responsible for many output errors and show that fixing these inputs reduces factual errors by between 52% and 76% (depending on the model). In addition, we observe that models struggle in processing tabular inputs that are structured in a non-standard way, particularly when the input lacks distinct row and column values or when the column headers are not correctly mapped to corresponding values.
Anthology ID:
2024.naacl-long.408
Volume:
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7343–7369
Language:
URL:
https://aclanthology.org/2024.naacl-long.408
DOI:
Bibkey:
Cite (ACL):
Barkavi Sundararajan, Yaji Sripada, and Ehud Reiter. 2024. Improving Factual Accuracy of Neural Table-to-Text Output by Addressing Input Problems in ToTTo. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7343–7369, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
Improving Factual Accuracy of Neural Table-to-Text Output by Addressing Input Problems in ToTTo (Sundararajan et al., NAACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.naacl-long.408.pdf
Copyright:
 2024.naacl-long.408.copyright.pdf