A-TIP: Attribute-aware Text Infilling via Pre-trained Language Model

Dongyuan Li, Jingyi You, Kotaro Funakoshi, Manabu Okumura


Abstract
Text infilling aims to restore incomplete texts by filling in blanks, which has attracted more attention recently because of its wide application in ancient text restoration and text rewriting. However, attribute- aware text infilling is yet to be explored, and existing methods seldom focus on the infilling length of each blank or the number/location of blanks. In this paper, we propose an Attribute-aware Text Infilling method via a Pre-trained language model (A-TIP), which contains a text infilling component and a plug- and-play discriminator. Specifically, we first design a unified text infilling component with modified attention mechanisms and intra- and inter-blank positional encoding to better perceive the number of blanks and the infilling length for each blank. Then, we propose a plug-and-play discriminator to guide generation towards the direction of improving attribute relevance without decreasing text fluency. Finally, automatic and human evaluations on three open-source datasets indicate that A-TIP achieves state-of- the-art performance compared with all baselines.
Anthology ID:
2022.coling-1.511
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
5857–5869
Language:
URL:
https://aclanthology.org/2022.coling-1.511
DOI:
Bibkey:
Cite (ACL):
Dongyuan Li, Jingyi You, Kotaro Funakoshi, and Manabu Okumura. 2022. A-TIP: Attribute-aware Text Infilling via Pre-trained Language Model. In Proceedings of the 29th International Conference on Computational Linguistics, pages 5857–5869, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
A-TIP: Attribute-aware Text Infilling via Pre-trained Language Model (Li et al., COLING 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.coling-1.511.pdf
Data
ROCStories