Conditional Generation of Temporally-ordered Event Sequences

Shih-Ting Lin, Nathanael Chambers, Greg Durrett


Abstract
Models of narrative schema knowledge have proven useful for a range of event-related tasks, but they typically do not capture the temporal relationships between events. We propose a single model that addresses both temporal ordering, sorting given events into the order they occurred, and event infilling, predicting new events which fit into an existing temporally-ordered sequence. We use a BART-based conditional generation model that can capture both temporality and common event co-occurrence, meaning it can be flexibly applied to different tasks in this space. Our model is trained as a denoising autoencoder: we take temporally-ordered event sequences, shuffle them, delete some events, and then attempt to recover the original event sequence. This task teaches the model to make inferences given incomplete knowledge about the events in an underlying scenario. On the temporal ordering task, we show that our model is able to unscramble event sequences from existing datasets without access to explicitly labeled temporal training data, outperforming both a BERT-based pairwise model and a BERT-based pointer network. On event infilling, human evaluation shows that our model is able to generate events that fit better temporally into the input events when compared to GPT-2 story completion models.
Anthology ID:
2021.acl-long.555
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7142–7157
Language:
URL:
https://aclanthology.org/2021.acl-long.555
DOI:
10.18653/v1/2021.acl-long.555
Bibkey:
Cite (ACL):
Shih-Ting Lin, Nathanael Chambers, and Greg Durrett. 2021. Conditional Generation of Temporally-ordered Event Sequences. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7142–7157, Online. Association for Computational Linguistics.
Cite (Informal):
Conditional Generation of Temporally-ordered Event Sequences (Lin et al., ACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-long.555.pdf
Video:
 https://aclanthology.org/2021.acl-long.555.mp4
Data
MC-TACO