Ad-hoc Document Retrieval using Weak-Supervision with BERT and GPT2

Yosi Mass, Haggai Roitman


Abstract
We describe a weakly-supervised method for training deep learning models for the task of ad-hoc document retrieval. Our method is based on generative and discriminative models that are trained using weak-supervision just from the documents in the corpus. We present an end-to-end retrieval system that starts with traditional information retrieval methods, followed by two deep learning re-rankers. We evaluate our method on three different datasets: a COVID-19 related scientific literature dataset and two news datasets. We show that our method outperforms state-of-the-art methods; this without the need for the expensive process of manually labeling data.
Anthology ID:
2020.emnlp-main.343
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4191–4197
Language:
URL:
https://aclanthology.org/2020.emnlp-main.343
DOI:
10.18653/v1/2020.emnlp-main.343
Bibkey:
Cite (ACL):
Yosi Mass and Haggai Roitman. 2020. Ad-hoc Document Retrieval using Weak-Supervision with BERT and GPT2. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4191–4197, Online. Association for Computational Linguistics.
Cite (Informal):
Ad-hoc Document Retrieval using Weak-Supervision with BERT and GPT2 (Mass & Roitman, EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.343.pdf
Video:
 https://slideslive.com/38939294