Hierarchical Neural Story Generation

Angela Fan, Mike Lewis, Yann Dauphin


Abstract
We explore story generation: creative systems that can build coherent and fluent passages of text about a topic. We collect a large dataset of 300K human-written stories paired with writing prompts from an online forum. Our dataset enables hierarchical story generation, where the model first generates a premise, and then transforms it into a passage of text. We gain further improvements with a novel form of model fusion that improves the relevance of the story to the prompt, and adding a new gated multi-scale self-attention mechanism to model long-range context. Experiments show large improvements over strong baselines on both automated and human evaluations. Human judges prefer stories generated by our approach to those from a strong non-hierarchical model by a factor of two to one.
Anthology ID:
P18-1082
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
889–898
Language:
URL:
https://aclanthology.org/P18-1082
DOI:
10.18653/v1/P18-1082
Bibkey:
Copy Citation:
PDF:
https://aclanthology.org/P18-1082.pdf
Note:
 P18-1082.Notes.pdf
Video:
 https://vimeo.com/285801163
Code
 pytorch/fairseq +  additional community code
Data
WritingPrompts