Max Nelson
Author directory2026
Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines
Aviral Joshi | Hanoz Bhathena | Max Nelson | Saket Sharma
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Aviral Joshi | Hanoz Bhathena | Max Nelson | Saket Sharma
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Retrieval-augmented generation (RAG) is hard to monitor in production: exhaustive relevance labels do not exist for non-stationary multi-million-passage corpora that re-index in real time. As a result, retrieval quality is generally understudied and often deprioritised in favour of generation-oriented metrics. In this work, we propose auditing retrieval coverage by probing for evidence of missing documents rather than enumerating every relevant one. Our method Re:CAP (REtrieval Coverage Audit by iterative Probing) is a reference-free audit loop applied to a deployed RAG pipeline’s initial answer and retrieved context: it identifies the topics already covered, generates probing questions for plausibly missing topics, retrieves candidate documents, and applies an LLM-as-judge to retain only those that introduce previously-unretrieved information. On four public benchmarks, Re:CAP recovers 9–29% of gold labels that flat BM25 top-500 cannot reach, rising to 48% on TREC-COVID. On MuSiQue Re:CAP beats flat hybrid top-500 by +12.9 pp on recall at less than half the document budget. An ensemble BM25, dense, and hybrid baseline (top-500 each) still leaves out 21.2% of gold docs on TREC-COVID that Re:CAP recovers; human annotators judge that 78.9% of those structurally distinct documents add new information to the baseline answer (Fleiss κ = 0.79, n = 123), and 73.9% on live production traffic (n = 180). End-to-end recall is reproducible to within ±1% across three independent runs, making Re:CAP a stable instrument for periodic retrieval audits.
2020
Joint learning of constraint weights and gradient inputs in Gradient Symbolic Computation with constrained optimization
Max Nelson
Proceedings of the 17th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology
Max Nelson
Proceedings of the 17th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology
This paper proposes a method for the joint optimization of constraint weights and symbol activations within the Gradient Symbolic Computation (GSC) framework. The set of grammars representable in GSC is proven to be a subset of those representable with lexically-scaled faithfulness constraints. This fact is then used to recast the problem of learning constraint weights and symbol activations in GSC as a quadratically-constrained version of learning lexically-scaled faithfulness grammars. This results in an optimization problem that can be solved using Sequential Quadratic Programming.
Phonotactic learning with neural language models
Connor Mayer | Max Nelson
Proceedings of the Society for Computation in Linguistics 2020
Connor Mayer | Max Nelson
Proceedings of the Society for Computation in Linguistics 2020
Probing RNN Encoder-Decoder Generalization of Subregular Functions using Reduplication
Max Nelson | Hossep Dolatian | Jonathan Rawski | Brandon Prickett
Proceedings of the Society for Computation in Linguistics 2020
Max Nelson | Hossep Dolatian | Jonathan Rawski | Brandon Prickett
Proceedings of the Society for Computation in Linguistics 2020