Huiming Yang

Author directory

2026

Large language models struggle with spatial and geometric reasoning. Existing remedies either fine-tune on answers, leaving geometric structure implicit, or attach a symbolic solver at inference, which is brittle under imperfect parser coverage. We propose **DGCL** (*Differentiable Geometric Constraint Layer*), a training-time objective for a typed family of spatial and geometric relations (e.g., parallelism, frame-relative left/right, vertical order) that supervises the LLM’s hidden states with a differentiable Huber residual over parser-proposed candidate constraints, a coverage regularizer, and a learned selector. Training requires a frozen constraint parser; inference does not: it is a single LLM forward pass over the text prompt plus one learned ‘[GEO]‘ token, with no parser, solver, external tool, or auxiliary head. We evaluate on three auditable datasets (planar geometry, language-grounded 3D relations, and a firewalled zero-touch counterfactual split) with 10,278 test instances over 5 seeds. We are explicit about attribution: a strong constraint-supervised control (A2c), which trains the selector and coverage objective on gold candidates but blocks the residual gradient from the LLM, already captures most of the improvement over plain fine-tuning (up to +26 points). Our isolated contribution is that *coupling* the differentiable residual gradient into the LLM adds a *further, stable* +4.0 to +5.2 points over this strong control across all five settings (seven pre-registered gates, Holm-corrected p<0.001), at 1.06× a fine-tuned baseline’s latency. On full parser coverage a strong tool baseline marginally exceeds DGCL; DGCL’s advantage is robustness under partial or missing coverage, not superiority over oracle solvers.