Adversarial Decomposition of Text Representation

Alexey Romanov, Anna Rumshisky, Anna Rogers, David Donahue


Abstract
In this paper, we present a method for adversarial decomposition of text representation. This method can be used to decompose a representation of an input sentence into several independent vectors, each of them responsible for a specific aspect of the input sentence. We evaluate the proposed method on two case studies: the conversion between different social registers and diachronic language change. We show that the proposed method is capable of fine-grained controlled change of these aspects of the input sentence. It is also learning a continuous (rather than categorical) representation of the style of the sentence, which is more linguistically realistic. The model uses adversarial-motivational training and includes a special motivational loss, which acts opposite to the discriminator and encourages a better decomposition. Furthermore, we evaluate the obtained meaning embeddings on a downstream task of paraphrase detection and show that they significantly outperform the embeddings of a regular autoencoder.
Anthology ID:
N19-1088
Volume:
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
815–825
Language:
URL:
https://aclanthology.org/N19-1088
DOI:
10.18653/v1/N19-1088
Bibkey:
Cite (ACL):
Alexey Romanov, Anna Rumshisky, Anna Rogers, and David Donahue. 2019. Adversarial Decomposition of Text Representation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 815–825, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Adversarial Decomposition of Text Representation (Romanov et al., NAACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/N19-1088.pdf
Code
 text-machine-lab/adversarial_decomposition +  additional community code