A BERT-based Dual Embedding Model for Chinese Idiom Prediction

Minghuan Tan, Jing Jiang


Abstract
Chinese idioms are special fixed phrases usually derived from ancient stories, whose meanings are oftentimes highly idiomatic and non-compositional. The Chinese idiom prediction task is to select the correct idiom from a set of candidate idioms given a context with a blank. We propose a BERT-based dual embedding model to encode the contextual words as well as to learn dual embeddings of the idioms. Specifically, we first match the embedding of each candidate idiom with the hidden representation corresponding to the blank in the context. We then match the embedding of each candidate idiom with the hidden representations of all the tokens in the context thorough context pooling. We further propose to use two separate idiom embeddings for the two kinds of matching. Experiments on a recently released Chinese idiom cloze test dataset show that our proposed method performs better than the existing state of the art. Ablation experiments also show that both context pooling and dual embedding contribute to the improvement of performance.
Anthology ID:
2020.coling-main.113
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Donia Scott, Nuria Bel, Chengqing Zong
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
1312–1322
Language:
URL:
https://aclanthology.org/2020.coling-main.113
DOI:
10.18653/v1/2020.coling-main.113
Bibkey:
Cite (ACL):
Minghuan Tan and Jing Jiang. 2020. A BERT-based Dual Embedding Model for Chinese Idiom Prediction. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1312–1322, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
A BERT-based Dual Embedding Model for Chinese Idiom Prediction (Tan & Jiang, COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-main.113.pdf
Code
 VisualJoyce/ChengyuBERT
Data
ChID