Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity

Sheshera Mysore, Arman Cohan, Tom Hope


Abstract
We present a new scientific document similarity model based on matching fine-grained aspects of texts. To train our model, we exploit a naturally-occurring source of supervision: sentences in the full-text of papers that cite multiple papers together (co-citations). Such co-citations not only reflect close paper relatedness, but also provide textual descriptions of how the co-cited papers are related. This novel form of textual supervision is used for learning to match aspects across papers. We develop multi-vector representations where vectors correspond to sentence-level aspects of documents, and present two methods for aspect matching: (1) A fast method that only matches single aspects, and (2) a method that makes sparse multiple matches with an Optimal Transport mechanism that computes an Earth Mover’s Distance between aspects. Our approach improves performance on document similarity tasks in four datasets. Further, our fast single-match method achieves competitive results, paving the way for applying fine-grained similarity to large scientific corpora.
Anthology ID:
2022.naacl-main.331
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4453–4470
Language:
URL:
https://aclanthology.org/2022.naacl-main.331
DOI:
10.18653/v1/2022.naacl-main.331
Bibkey:
Cite (ACL):
Sheshera Mysore, Arman Cohan, and Tom Hope. 2022. Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4453–4470, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity (Mysore et al., NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-main.331.pdf
Video:
 https://aclanthology.org/2022.naacl-main.331.mp4
Code
 allenai/aspire
Data
TREC-COVID