Building Hierarchically Disentangled Language Models for Text Generation with Named Entities

Yash Agarwal, Devansh Batra, Ganesh Bagler


Abstract
Named entities pose a unique challenge to traditional methods of language modeling. While several domains are characterised with a high proportion of named entities, the occurrence of specific entities varies widely. Cooking recipes, for example, contain a lot of named entities — viz. ingredients, cooking techniques (also called processes), and utensils. However, some ingredients occur frequently within the instructions while most occur rarely. In this paper, we build upon the previous work done on language models developed for text with named entities by introducing a Hierarchically Disentangled Model. Training is divided into multiple branches with each branch producing a model with overlapping subsets of vocabulary. We found the existing datasets insufficient to accurately judge the performance of the model. Hence, we have curated 158,473 cooking recipes from several publicly available online sources. To reliably derive the entities within this corpus, we employ a combination of Named Entity Recognition (NER) as well as an unsupervised method of interpretation using dependency parsing and POS tagging, followed by a further cleaning of the dataset. This unsupervised interpretation models instructions as action graphs and is specific to the corpus of cooking recipes, unlike NER which is a general method applicable to all corpora. To delve into the utility of our language model, we apply it to tasks such as graph-to-text generation and ingredients-to-recipe generation, comparing it to previous state-of-the-art baselines. We make our dataset (including annotations and processed action graphs) available for use, considering their potential use cases for language modeling and text generation research.
Anthology ID:
2020.coling-main.3
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Donia Scott, Nuria Bel, Chengqing Zong
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
26–38
Language:
URL:
https://aclanthology.org/2020.coling-main.3
DOI:
10.18653/v1/2020.coling-main.3
Bibkey:
Cite (ACL):
Yash Agarwal, Devansh Batra, and Ganesh Bagler. 2020. Building Hierarchically Disentangled Language Models for Text Generation with Named Entities. In Proceedings of the 28th International Conference on Computational Linguistics, pages 26–38, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
Building Hierarchically Disentangled Language Models for Text Generation with Named Entities (Agarwal et al., COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-main.3.pdf
Data
Recipe1M+