Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning

Fei Wang, Zhewei Xu, Pedro Szekely, Muhao Chen


Abstract
Controlled table-to-text generation seeks to generate natural language descriptions for highlighted subparts of a table. Previous SOTA systems still employ a sequence-to-sequence generation method, which merely captures the table as a linear structure and is brittle when table layouts change. We seek to go beyond this paradigm by (1) effectively expressing the relations of content pieces in the table, and (2) making our model robust to content-invariant structural transformations. Accordingly, we propose an equivariance learning framework, which encodes tables with a structure-aware self-attention mechanism. This prunes the full self-attention structure into an order-invariant graph attention that captures the connected graph structure of cells belonging to the same row or column, and it differentiates between relevant cells and irrelevant cells from the structural perspective. Our framework also modifies the positional encoding mechanism to preserve the relative position of tokens in the same cell but enforce position invariance among different cells. Our technology is free to be plugged into existing table-to-text generation models, and has improved T5-based models to offer better performance on ToTTo and HiTab. Moreover, on a harder version of ToTTo, we preserve promising performance, while previous SOTA systems, even with transformation-based data augmentation, have seen significant performance drops.
Anthology ID:
2022.naacl-main.371
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5037–5048
Language:
URL:
https://aclanthology.org/2022.naacl-main.371
DOI:
10.18653/v1/2022.naacl-main.371
Bibkey:
Cite (ACL):
Fei Wang, Zhewei Xu, Pedro Szekely, and Muhao Chen. 2022. Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5037–5048, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning (Wang et al., NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-main.371.pdf
Code
 luka-group/lattice