DIFM:An effective deep interaction and fusion model for sentence matching

Jiang Kexin, Zhao Yahui, Cui Rongyi


Abstract
“Natural language sentence matching is the task of comparing two sentences and identifying the relationship between them. It has a wide range of applications in natural language processing tasks such as reading comprehension, question and answer systems. The main approach is to compute the interaction between text representations and sentence pairs through an attention mechanism, which can extract the semantic information between sentence pairs well. However, this kind of methods fail to capture deep semantic information and effectively fuse the semantic information of the sentence. To solve this problem, we propose a sentence matching method based on deep interaction and fusion. We first use pre-trained word vectors Glove and characterlevel word vectors to obtain word embedding representations of the two sentences. In the encoding layer, we use bidirectional LSTM to encode the sentence pairs. In the interaction layer, we initially fuse the information of the sentence pairs to obtain low-level semantic information; at the same time, we use the bi-directional attention in the machine reading comprehension model and self-attention to obtain the high-level semantic information. We use a heuristic fusion function to fuse the low-level semantic information and the high-level semantic information to obtain the final semantic information, and finally we use the convolutional neural network to predict the answer. We evaluate our model on two tasks: text implication recognition and paraphrase recognition. We conducted experiments on the SNLI datasets for the recognizing textual entailment task, the Quora dataset for the paraphrase recognition task. The experimental results show that the proposed algorithm can effectively fuse different semantic information that verify the effectiveness of the algorithm on sentence matching tasks.”
Anthology ID:
2022.ccl-1.66
Volume:
Proceedings of the 21st Chinese National Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Nanchang, China
Editors:
Maosong Sun (孙茂松), Yang Liu (刘洋), Wanxiang Che (车万翔), Yang Feng (冯洋), Xipeng Qiu (邱锡鹏), Gaoqi Rao (饶高琦), Yubo Chen (陈玉博)
Venue:
CCL
SIG:
Publisher:
Chinese Information Processing Society of China
Note:
Pages:
738–747
Language:
English
URL:
https://aclanthology.org/2022.ccl-1.66
DOI:
Bibkey:
Cite (ACL):
Jiang Kexin, Zhao Yahui, and Cui Rongyi. 2022. DIFM:An effective deep interaction and fusion model for sentence matching. In Proceedings of the 21st Chinese National Conference on Computational Linguistics, pages 738–747, Nanchang, China. Chinese Information Processing Society of China.
Cite (Informal):
DIFM:An effective deep interaction and fusion model for sentence matching (Kexin et al., CCL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.ccl-1.66.pdf
Data
SNLI