Wolfgang Lehner


pdf bib
Extracting Operator Trees from Model Embeddings
Anja Reusch | Wolfgang Lehner
Proceedings of the 1st Workshop on Mathematical Natural Language Processing (MathNLP)

Transformer-based language models are able to capture several linguistic properties such as hierarchical structures like dependency or constituency trees. Whether similar structures for mathematics are extractable from language models has not yet been explored. This work aims to probe current state-of-the-art models for the extractability of Operator Trees from their contextualized embeddings using the structure probe designed by Hewitt and Manning. We release the code and our data set for future analysis.