Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction

Hirokazu Kiyomaru, Kazumasa Omura, Yugo Murawaki, Daisuke Kawahara, Sadao Kurohashi


Abstract
Typical event sequences are an important class of commonsense knowledge. Formalizing the task as the generation of a next event conditioned on a current event, previous work in event prediction employs sequence-to-sequence (seq2seq) models. However, what can happen after a given event is usually diverse, a fact that can hardly be captured by deterministic models. In this paper, we propose to incorporate a conditional variational autoencoder (CVAE) into seq2seq for its ability to represent diverse next events as a probabilistic distribution. We further extend the CVAE-based seq2seq with a reconstruction mechanism to prevent the model from concentrating on highly typical events. To facilitate fair and systematic evaluation of the diversity-aware models, we also extend existing evaluation datasets by tying each current event to multiple next events. Experiments show that the CVAE-based models drastically outperform deterministic models in terms of precision and that the reconstruction mechanism improves the recall of CVAE-based models without sacrificing precision.
Anthology ID:
D19-6014
Volume:
Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Simon Ostermann, Sheng Zhang, Michael Roth, Peter Clark
Venue:
WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
113–122
Language:
URL:
https://aclanthology.org/D19-6014
DOI:
10.18653/v1/D19-6014
Bibkey:
Cite (ACL):
Hirokazu Kiyomaru, Kazumasa Omura, Yugo Murawaki, Daisuke Kawahara, and Sadao Kurohashi. 2019. Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction. In Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing, pages 113–122, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction (Kiyomaru et al., 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-6014.pdf