Incorporating Stylistic Lexical Preferences in Generative Language Models

Hrituraj Singh, Gaurav Verma, Balaji Vasan Srinivasan


Abstract
While recent advances in language modeling has resulted in powerful generation models, their generation style remains implicitly dependent on the training data and can not emulate a specific target style. Leveraging the generative capabilities of a transformer-based language models, we present an approach to induce certain target-author attributes by incorporating continuous multi-dimensional lexical preferences of an author into generative language models. We introduce rewarding strategies in a reinforcement learning framework that encourages the use of words across multiple categorical dimensions, to varying extents. Our experiments demonstrate that the proposed approach can generate text that distinctively aligns with a given target author’s lexical style. We conduct quantitative and qualitative comparisons with competitive and relevant baselines to illustrate the benefits of the proposed approach.
Anthology ID:
2020.findings-emnlp.96
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2020
Month:
November
Year:
2020
Address:
Online
Editors:
Trevor Cohn, Yulan He, Yang Liu
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1074–1079
Language:
URL:
https://aclanthology.org/2020.findings-emnlp.96
DOI:
10.18653/v1/2020.findings-emnlp.96
Bibkey:
Cite (ACL):
Hrituraj Singh, Gaurav Verma, and Balaji Vasan Srinivasan. 2020. Incorporating Stylistic Lexical Preferences in Generative Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1074–1079, Online. Association for Computational Linguistics.
Cite (Informal):
Incorporating Stylistic Lexical Preferences in Generative Language Models (Singh et al., Findings 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.findings-emnlp.96.pdf
Data
WebText