Generation of Hip-Hop Lyrics with Hierarchical Modeling and Conditional Templates

Enrique Manjavacas, Mike Kestemont, Folgert Karsdorp


Abstract
This paper addresses Hip-Hop lyric generation with conditional Neural Language Models. We develop a simple yet effective mechanism to extract and apply conditional templates from text snippets, and show—on the basis of a large-scale crowd-sourced manual evaluation—that these templates significantly improve the quality and realism of the generated snippets. Importantly, the proposed approach enables end-to-end training, targeting formal properties of text such as rhythm and rhyme, which are central characteristics of rap texts. Additionally, we explore how generating text at different scales (e.g. character-level or word-level) affects the quality of the output. We find that a hybrid form—a hierarchical model that aims to integrate Language Modeling at both word and character-level scales—yields significant improvements in text quality, yet surprisingly, cannot exploit conditional templates to their fullest extent. Our findings highlight that text generation models based on Recurrent Neural Networks (RNN) are sensitive to the modeling scale and call for further research on the observed differences in effectiveness of the conditioning mechanism at different scales.
Anthology ID:
W19-8638
Volume:
Proceedings of the 12th International Conference on Natural Language Generation
Month:
October–November
Year:
2019
Address:
Tokyo, Japan
Editors:
Kees van Deemter, Chenghua Lin, Hiroya Takamura
Venue:
INLG
SIG:
SIGGEN
Publisher:
Association for Computational Linguistics
Note:
Pages:
301–310
Language:
URL:
https://aclanthology.org/W19-8638
DOI:
10.18653/v1/W19-8638
Bibkey:
Cite (ACL):
Enrique Manjavacas, Mike Kestemont, and Folgert Karsdorp. 2019. Generation of Hip-Hop Lyrics with Hierarchical Modeling and Conditional Templates. In Proceedings of the 12th International Conference on Natural Language Generation, pages 301–310, Tokyo, Japan. Association for Computational Linguistics.
Cite (Informal):
Generation of Hip-Hop Lyrics with Hierarchical Modeling and Conditional Templates (Manjavacas et al., INLG 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-8638.pdf