Towards Structure-aware Paraphrase Identification with Phrase Alignment Using Sentence Encoders

Qiwei Peng, David Weir, Julie Weeds


Abstract
Previous works have demonstrated the effectiveness of utilising pre-trained sentence encoders based on their sentence representations for meaning comparison tasks. Though such representations are shown to capture hidden syntax structures, the direct similarity comparison between them exhibits weak sensitivity to word order and structural differences in given sentences. A single similarity score further makes the comparison process hard to interpret. Therefore, we here propose to combine sentence encoders with an alignment component by representing each sentence as a list of predicate-argument spans (where their span representations are derived from sentence encoders), and decomposing the sentence-level meaning comparison into the alignment between their spans for paraphrase identification tasks. Empirical results show that the alignment component brings in both improved performance and interpretability for various sentence encoders. After closer investigation, the proposed approach indicates increased sensitivity to structural difference and enhanced ability to distinguish non-paraphrases with high lexical overlap.
Anthology ID:
2022.coling-1.361
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
4113–4123
Language:
URL:
https://aclanthology.org/2022.coling-1.361
DOI:
Bibkey:
Cite (ACL):
Qiwei Peng, David Weir, and Julie Weeds. 2022. Towards Structure-aware Paraphrase Identification with Phrase Alignment Using Sentence Encoders. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4113–4123, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
Towards Structure-aware Paraphrase Identification with Phrase Alignment Using Sentence Encoders (Peng et al., COLING 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.coling-1.361.pdf
Data
GLUEPAWS