Cydex: Neural Search Infrastructure for the Scholarly Literature

Shane Ding, Edwin Zhang, Jimmy Lin


Abstract
Cydex is a platform that provides neural search infrastructure for domain-specific scholarly literature. The platform represents an abstraction of Covidex, our recently developed full-stack open-source search engine for the COVID-19 Open Research Dataset (CORD-19) from AI2. While Covidex takes advantage of the latest best practices for keyword search using the popular Lucene search library as well as state-of-the-art neural ranking models using T5, parts of the system were hard coded to only work with CORD-19. This paper describes our efforts to generalize Covidex into Cydex, which can be applied to scholarly literature in different domains. By decoupling corpus-specific configurations from the frontend implementation, we are able to demonstrate the generality of Cydex on two very different corpora: the ACL Anthology and a collection of hydrology abstracts. Our platform is entirely open source and available at cydex.ai.
Anthology ID:
2020.sdp-1.19
Volume:
Proceedings of the First Workshop on Scholarly Document Processing
Month:
November
Year:
2020
Address:
Online
Editors:
Muthu Kumar Chandrasekaran, Anita de Waard, Guy Feigenblat, Dayne Freitag, Tirthankar Ghosal, Eduard Hovy, Petr Knoth, David Konopnicki, Philipp Mayr, Robert M. Patton, Michal Shmueli-Scheuer
Venue:
sdp
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
168–173
Language:
URL:
https://aclanthology.org/2020.sdp-1.19
DOI:
10.18653/v1/2020.sdp-1.19
Bibkey:
Cite (ACL):
Shane Ding, Edwin Zhang, and Jimmy Lin. 2020. Cydex: Neural Search Infrastructure for the Scholarly Literature. In Proceedings of the First Workshop on Scholarly Document Processing, pages 168–173, Online. Association for Computational Linguistics.
Cite (Informal):
Cydex: Neural Search Infrastructure for the Scholarly Literature (Ding et al., sdp 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.sdp-1.19.pdf
Video:
 https://slideslive.com/38940734
Data
CORD-19TREC-COVID