Benjamin E Nye
Author directory2026
NaturalPRISM: A Natural Language Premise Retrieval Task for Research-level Intermediate Proof States
Harris Proctor | Li An | Austin LaHue | Michael Burr | Vinita Gangaram Jansari | Carlos Toxtli Hernandez | Luis David Garcia Puente | Benjamin E Nye
The Proceedings of the 4th Workshop on Mathematical Natural Language Processing (MathNLP 2026)
Harris Proctor | Li An | Austin LaHue | Michael Burr | Vinita Gangaram Jansari | Carlos Toxtli Hernandez | Luis David Garcia Puente | Benjamin E Nye
The Proceedings of the 4th Workshop on Mathematical Natural Language Processing (MathNLP 2026)
As AI use progresses in research-level mathematics, automated information retrieval systems for constructing relevant prompts are increasingly important. Existing benchmarks and datasets for identifying which prior results are necessary to prove a statement do not align well with current workflows in which a natural language proof is in progress, but it is unclear what the next step(s) should be.
We introduce NaturalPRISM, a benchmark task and dataset containing 1.7 million pairs of intermediate proof states and the premise invoked in the next step of the proof. All pairs are extracted from natural language research-level mathematics papers published on arXiv, and span all 32 subject categories.
We evaluate the effect of fine-tuning retrieval models for domain-specific vs. domain-agnostic use cases, and find that the combination of sufficient training data with alignment between the training and testing distributions allows smaller specialized models to dramatically outperform larger general models.
We introduce NaturalPRISM, a benchmark task and dataset containing 1.7 million pairs of intermediate proof states and the premise invoked in the next step of the proof. All pairs are extracted from natural language research-level mathematics papers published on arXiv, and span all 32 subject categories.
We evaluate the effect of fine-tuning retrieval models for domain-specific vs. domain-agnostic use cases, and find that the combination of sufficient training data with alignment between the training and testing distributions allows smaller specialized models to dramatically outperform larger general models.
Which Mathematical Context Supports Target-Aligned Conjecture Generation? A Paired Ablation Study
Austin LaHue | Michael Burr | Luis David Garcia Puente | Vinita Gangaram Jansari | Benjamin E Nye | Carlos Toxtli Hernandez
The Proceedings of the 4th Workshop on Mathematical Natural Language Processing (MathNLP 2026)
Austin LaHue | Michael Burr | Luis David Garcia Puente | Vinita Gangaram Jansari | Benjamin E Nye | Carlos Toxtli Hernandez
The Proceedings of the 4th Workshop on Mathematical Natural Language Processing (MathNLP 2026)
Generating mathematical conjectures from natural-language context is challenging because theorem-relevant information is distributed across multiple forms of mathematical context, including definitions, objects, assumptions, and constructions. We study which of these contextual signals help a model recover the specific claim supported by the surrounding setup, using 61 main theorems from 12 papers across four mathematical domains. For each theorem, we redact the result and represent the remaining setup using six context components. A paired ablation design compares full context, six leave-one-out conditions, and six single-component conditions, yielding 793 generated conjectures. Context components were largely complementary: removing one component usually had little effect, whereas retaining only one substantially reduced expected-theorem alignment. Local Mathematical Objects provided the strongest standalone signal, while Structural Constraints and Conditions produced the largest alignment decrease when removed. These findings suggest that target-aligned conjecture generation depends on combining information that identifies relevant objects with information that constrains admissible claims.