Knowledge Base Index Compression via Dimensionality and Precision Reduction

Vilém Zouhar, Marius Mosbach, Miaoran Zhang, Dietrich Klakow


Abstract
Recently neural network based approaches to knowledge-intensive NLP tasks, such as question answering, started to rely heavily on the combination of neural retrievers and readers. Retrieval is typically performed over a large textual knowledge base (KB) which requires significant memory and compute resources, especially when scaled up. On HotpotQA we systematically investigate reducing the size of the KB index by means of dimensionality (sparse random projections, PCA, autoencoders) and numerical precision reduction. Our results show that PCA is an easy solution that requires very little data and is only slightly worse than autoencoders, which are less stable. All methods are sensitive to pre- and post-processing and data should always be centered and normalized both before and after dimension reduction. Finally, we show that it is possible to combine PCA with using 1bit per dimension. Overall we achieve (1) 100× compression with 75%, and (2) 24× compression with 92% original retrieval performance.
Anthology ID:
2022.spanlp-1.5
Volume:
Proceedings of the 1st Workshop on Semiparametric Methods in NLP: Decoupling Logic from Knowledge
Month:
May
Year:
2022
Address:
Dublin, Ireland and Online
Editors:
Rajarshi Das, Patrick Lewis, Sewon Min, June Thai, Manzil Zaheer
Venue:
SpaNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
41–53
Language:
URL:
https://aclanthology.org/2022.spanlp-1.5
DOI:
10.18653/v1/2022.spanlp-1.5
Bibkey:
Cite (ACL):
Vilém Zouhar, Marius Mosbach, Miaoran Zhang, and Dietrich Klakow. 2022. Knowledge Base Index Compression via Dimensionality and Precision Reduction. In Proceedings of the 1st Workshop on Semiparametric Methods in NLP: Decoupling Logic from Knowledge, pages 41–53, Dublin, Ireland and Online. Association for Computational Linguistics.
Cite (Informal):
Knowledge Base Index Compression via Dimensionality and Precision Reduction (Zouhar et al., SpaNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.spanlp-1.5.pdf
Video:
 https://aclanthology.org/2022.spanlp-1.5.mp4
Code
 zouharvi/kb-shrink
Data
HotpotQANatural Questions