Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling

Timothee Mickus, Denis Paperno, Matthieu Constant


Abstract
Defining words in a textual context is a useful task both for practical purposes and for gaining insight into distributed word representations. Building on the distributional hypothesis, we argue here that the most natural formalization of definition modeling is to treat it as a sequence-to-sequence task, rather than a word-to-sequence task: given an input sequence with a highlighted word, generate a contextually appropriate definition for it. We implement this approach in a Transformer-based sequence-to-sequence model. Our proposal allows to train contextualization and definition generation in an end-to-end fashion, which is a conceptual improvement over earlier works. We achieve state-of-the-art results both in contextual and non-contextual definition modeling.
Anthology ID:
W19-6201
Volume:
Proceedings of the First NLPL Workshop on Deep Learning for Natural Language Processing
Month:
September
Year:
2019
Address:
Turku, Finland
Editors:
Joakim Nivre, Leon Derczynski, Filip Ginter, Bjørn Lindi, Stephan Oepen, Anders Søgaard, Jörg Tidemann
Venue:
NoDaLiDa
SIG:
Publisher:
Linköping University Electronic Press
Note:
Pages:
1–11
Language:
URL:
https://aclanthology.org/W19-6201
DOI:
Bibkey:
Cite (ACL):
Timothee Mickus, Denis Paperno, and Matthieu Constant. 2019. Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling. In Proceedings of the First NLPL Workshop on Deep Learning for Natural Language Processing, pages 1–11, Turku, Finland. Linköping University Electronic Press.
Cite (Informal):
Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling (Mickus et al., NoDaLiDa 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-6201.pdf