Aaditya Anand Thokal

Author directory

2026

Deterministic, parameter-free post-hoc correction rules are an attractive way to address numerical errors in financial question answering: they are simple, cheap, and fully auditable. We ask whether such rules actually behave like general-purpose correctors, or whether their effect is contingent on properties of the dataset and model output distribution they are applied to. We evaluate a two-rule deterministic verification layer (DVL) — scale_mul100 and scale_div100 — on FinQA (n=873) and TAT-QA (n=718) using Mistral-7B-Instruct-v0.3. This same, fixed model-plus-rule configuration produces a striking cross-dataset divergence: harmful on FinQA (37.34%→16.04%, −21.31pp, McNemar p=1.76×10⁻³⁹) and beneficial on TAT-QA (26.32%→30.64%, +4.32pp, p=5.15×10⁻³). A rule-level transition analysis shows scale_mul100 is highly precise on TAT-QA (87.95%) but overwhelmingly imprecise on FinQA (3.39%), while scale_div100 produces zero observed corrections and only harm in both settings; removing it raises TAT-QA accuracy further, to 35.10%. We do not find evidence that these rules correct arithmetic, retrieval, or semantic reasoning failures. We conclude that this specific pair of scaling rules is better understood as a targeted, dataset-contingent representation normalizer than as a general reasoning corrector, and we characterize the boundary conditions under which it helps or hurts.
Search
Co-authors
    Venues
    Fix author