Genki Osada
Author directory2026
Beyond Better Helpers: A Shared-Kernel Study of Symbolic Interfaces for Auditable LLM Algebra
Norio Kosaka | Genki Osada | Shinichi Higashino | Shuji Yamaguchi
The Proceedings of the 4th Workshop on Mathematical Natural Language Processing (MathNLP 2026)
Norio Kosaka | Genki Osada | Shinichi Higashino | Shuji Yamaguchi
The Proceedings of the 4th Workshop on Mathematical Natural Language Processing (MathNLP 2026)
Large language models can delegate algebra to external tools, but improvements attributed to a structured tool interface may instead come from the symbolic helper code hidden behind that interface. We isolate these factors on polynomial systems by holding the model, tasks, backend, and symbolic kernel fixed. Using Qwen3-8B, we compare text-only reasoning, unrestricted Python/SymPy generation, generated Python calling the same helper functions as the structured system, and a typed diagnosis–solve–check interface. On 36 tasks, the typed interface obtains 25/36 correct answers, compared with 14/36 for matched-helper code, 17/36 for raw SymPy, and 16/36 for text only. It also produces 23/36 automatically verified-correct answers versus 4/36 for the matched-helper control, and contains no execution-error runs versus 29/36. The largest separation occurs on exact finite-set tasks, where the typed interface solves 8/12 systems and both code conditions solve 0/12. Access to packaged helper functionality alone therefore does not reproduce the typed condition’s performance. The typed interface also removes the need to generate imports, API calls, control flow, and result-handling code, so the comparison measures end-to-end interface usability rather than mathematical reasoning or schema syntax in isolation. We interpret the result as a deployed-interface effect for one model and a development-informed task suite.