Cross-Modal Contextualized Hidden State Projection Method for Expanding of Taxonomic Graphs

Irina Nikishina, Alsu Vakhitova, Elena Tutubalina, Alexander Panchenko


Abstract
Taxonomy is a graph of terms organized hierarchically using is-a (hypernymy) relations. We suggest novel candidate-free task formulation for the taxonomy enrichment task. To solve the task, we leverage lexical knowledge from the pre-trained models to predict new words missing in the taxonomic resource. We propose a method that combines graph-, and text-based contextualized representations from transformer networks to predict new entries to the taxonomy. We have evaluated the method suggested for this task against text-only baselines based on BERT and fastText representations. The results demonstrate that incorporation of graph embedding is beneficial in the task of hyponym prediction using contextualized models. We hope the new challenging task will foster further research in automatic text graph construction methods.
Anthology ID:
2022.textgraphs-1.2
Volume:
Proceedings of TextGraphs-16: Graph-based Methods for Natural Language Processing
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Venue:
TextGraphs
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11–24
Language:
URL:
https://aclanthology.org/2022.textgraphs-1.2
DOI:
Bibkey:
Cite (ACL):
Irina Nikishina, Alsu Vakhitova, Elena Tutubalina, and Alexander Panchenko. 2022. Cross-Modal Contextualized Hidden State Projection Method for Expanding of Taxonomic Graphs. In Proceedings of TextGraphs-16: Graph-based Methods for Natural Language Processing, pages 11–24, Gyeongju, Republic of Korea. Association for Computational Linguistics.
Cite (Informal):
Cross-Modal Contextualized Hidden State Projection Method for Expanding of Taxonomic Graphs (Nikishina et al., TextGraphs 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.textgraphs-1.2.pdf