Dilinuer Aishan

Author directory

2026

Test-time training (TTT) adapts a frozen large language model (LLM) at inference time by updating prompt-specific parameters before generation. It is especially appealing for long-context reasoning. However, vanilla TTT yields only marginal gains on modern LLMs. We trace this bottleneck to two coupled failures: the inner loop is unprimed, because LLMs are trained as static endpoints rather than initialization points for test-time gradient adaptation; and the proxy objective is misaligned, because prompt-masked next-token prediction (PM-NTP) rewards prompt memorization rather than answer correctness. We propose PrimeTTT, a lightweight TTT method that casts this adaptation problem as meta-learning. A compact hypernetwork emits a prompt-specific LoRA initialization for the inner loop, trained with a MAML-style second-order objective: the inner loop descends the test-time-available PM-NTP loss, while the outer loss is answer-conditioned NTP on the gold answer and is backpropagated through the unrolled inner steps. Across out-of-distribution benchmarks disjoint from training, PrimeTTT consistently outperforms strong TTT baselines and transfers across model families and scales.