Learning Mathematical Properties of Integers

Maria Ryskina, Kevin Knight


Abstract
Embedding words in high-dimensional vector spaces has proven valuable in many natural language applications. In this work, we investigate whether similarly-trained embeddings of integers can capture concepts that are useful for mathematical applications. We probe the integer embeddings for mathematical knowledge, apply them to a set of numerical reasoning tasks, and show that by learning the representations from mathematical sequence data, we can substantially improve over number embeddings learned from English text corpora.
Anthology ID:
2021.blackboxnlp-1.30
Volume:
Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Jasmijn Bastings, Yonatan Belinkov, Emmanuel Dupoux, Mario Giulianelli, Dieuwke Hupkes, Yuval Pinter, Hassan Sajjad
Venue:
BlackboxNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
389–395
Language:
URL:
https://aclanthology.org/2021.blackboxnlp-1.30
DOI:
10.18653/v1/2021.blackboxnlp-1.30
Bibkey:
Cite (ACL):
Maria Ryskina and Kevin Knight. 2021. Learning Mathematical Properties of Integers. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 389–395, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Learning Mathematical Properties of Integers (Ryskina & Knight, BlackboxNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.blackboxnlp-1.30.pdf
Code
 ryskina/integer-embedding-tests +  additional community code