Multilevel Text Alignment with Cross-Document Attention

Xuhui Zhou, Nikolaos Pappas, Noah A. Smith


Abstract
Text alignment finds application in tasks such as citation recommendation and plagiarism detection. Existing alignment methods operate at a single, predefined level and cannot learn to align texts at, for example, sentence and document levels. We propose a new learning approach that equips previously established hierarchical attention encoders for representing documents with a cross-document attention component, enabling structural comparisons across different levels (document-to-document and sentence-to-document). Our component is weakly supervised from document pairs and can align at multiple levels. Our evaluation on predicting document-to-document relationships and sentence-to-document relationships on the tasks of citation recommendation and plagiarism detection shows that our approach outperforms previously established hierarchical, attention encoders based on recurrent and transformer contextualization that are unaware of structural correspondence between documents.
Anthology ID:
2020.emnlp-main.407
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Editors:
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5012–5025
Language:
URL:
https://aclanthology.org/2020.emnlp-main.407
DOI:
10.18653/v1/2020.emnlp-main.407
Bibkey:
Cite (ACL):
Xuhui Zhou, Nikolaos Pappas, and Noah A. Smith. 2020. Multilevel Text Alignment with Cross-Document Attention. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5012–5025, Online. Association for Computational Linguistics.
Cite (Informal):
Multilevel Text Alignment with Cross-Document Attention (Zhou et al., EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.407.pdf
Video:
 https://slideslive.com/38938779
Code
 XuhuiZhou/CDA
Data
S2ORC