Grouped-Attention for Content-Selection and Content-Plan Generation

Bayu Distiawan Trisedya, Xiaojie Wang, Jianzhong Qi, Rui Zhang, Qingjun Cui


Abstract
Content-planning is an essential part of data-to-text generation to determine the order of data mentioned in generated texts. Recent neural data-to-text generation models employ Pointer Networks to explicitly learn content-plan given a set of attributes as input. They use LSTM to encode the input, which assumes a sequential relationship in the input. This may be sub-optimal to encode a set of attributes, where the attributes have a composite structure: the attributes are disordered while each attribute value is an ordered list of tokens. We handle this problem by proposing a neural content-planner that can capture both local and global contexts of such a structure. Specifically, we propose a novel attention mechanism called GSC-attention. A key component of the GSC-attention is grouped-attention, which is token-level attention constrained within each input attribute that enables our proposed model captures both local and global context. Moreover, our content-planner explicitly learns content-selection, which is integrated into the content-planner to select the most important data to be included in the generated text via an attention masking procedure. Experimental results show that our model outperforms the competitors by 4.92%, 4.70%, and 16.56% in terms of Damerau-Levenshtein Distance scores on three real-world datasets.
Anthology ID:
2021.findings-emnlp.166
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2021
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1935–1944
Language:
URL:
https://aclanthology.org/2021.findings-emnlp.166
DOI:
10.18653/v1/2021.findings-emnlp.166
Bibkey:
Cite (ACL):
Bayu Distiawan Trisedya, Xiaojie Wang, Jianzhong Qi, Rui Zhang, and Qingjun Cui. 2021. Grouped-Attention for Content-Selection and Content-Plan Generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1935–1944, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Grouped-Attention for Content-Selection and Content-Plan Generation (Trisedya et al., Findings 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.findings-emnlp.166.pdf
Video:
 https://aclanthology.org/2021.findings-emnlp.166.mp4
Data
RotoWireWikiBio