Unsupervised Text Style Transfer with Content Embeddings

Keith Carlson, Allen Riddell, Daniel Rockmore


Abstract
The style transfer task (here style is used in a broad “authorial” sense with many aspects including register, sentence structure, and vocabulary choice) takes text input and rewrites it in a specified target style preserving the meaning, but altering the style of the source text to match that of the target. Much of the existing research on this task depends on the use of parallel datasets. In this work we employ recent results in unsupervised cross-lingual language modeling (XLM) and machine translation to effect style transfer while treating the input data as unaligned. First, we show that adding “content embeddings” to the XLM which capture human-specified groupings of subject matter can improve performance over the baseline model. Evaluation of style transfer has often relied on metrics designed for machine translation which have received criticism of their suitability for this task. As a second contribution, we propose the use of a suite of classical stylometrics as a useful complement for evaluation. We select a few such measures and include these in the analysis of our results.
Anthology ID:
2021.ranlp-1.27
Volume:
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
Month:
September
Year:
2021
Address:
Held Online
Venue:
RANLP
SIG:
Publisher:
INCOMA Ltd.
Note:
Pages:
226–233
Language:
URL:
https://aclanthology.org/2021.ranlp-1.27
DOI:
Bibkey:
Cite (ACL):
Keith Carlson, Allen Riddell, and Daniel Rockmore. 2021. Unsupervised Text Style Transfer with Content Embeddings. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 226–233, Held Online. INCOMA Ltd..
Cite (Informal):
Unsupervised Text Style Transfer with Content Embeddings (Carlson et al., RANLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.ranlp-1.27.pdf