Assessing the Stylistic Properties of Neurally Generated Text in Authorship Attribution

Enrique Manjavacas, Jeroen De Gussem, Walter Daelemans, Mike Kestemont


Abstract
Recent applications of neural language models have led to an increased interest in the automatic generation of natural language. However impressive, the evaluation of neurally generated text has so far remained rather informal and anecdotal. Here, we present an attempt at the systematic assessment of one aspect of the quality of neurally generated text. We focus on a specific aspect of neural language generation: its ability to reproduce authorial writing styles. Using established models for authorship attribution, we empirically assess the stylistic qualities of neurally generated text. In comparison to conventional language models, neural models generate fuzzier text, that is relatively harder to attribute correctly. Nevertheless, our results also suggest that neurally generated text offers more valuable perspectives for the augmentation of training data.
Anthology ID:
W17-4914
Volume:
Proceedings of the Workshop on Stylistic Variation
Month:
September
Year:
2017
Address:
Copenhagen, Denmark
Editors:
Julian Brooke, Thamar Solorio, Moshe Koppel
Venue:
Style-Var
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
116–125
Language:
URL:
https://aclanthology.org/W17-4914
DOI:
10.18653/v1/W17-4914
Bibkey:
Cite (ACL):
Enrique Manjavacas, Jeroen De Gussem, Walter Daelemans, and Mike Kestemont. 2017. Assessing the Stylistic Properties of Neurally Generated Text in Authorship Attribution. In Proceedings of the Workshop on Stylistic Variation, pages 116–125, Copenhagen, Denmark. Association for Computational Linguistics.
Cite (Informal):
Assessing the Stylistic Properties of Neurally Generated Text in Authorship Attribution (Manjavacas et al., Style-Var 2017)
Copy Citation:
PDF:
https://aclanthology.org/W17-4914.pdf