Jiajun Sun
Author directoryOther people with similar names: Jiajun Sun
2026
Latent Ordinal Evidence, Misaligned Outputs: Inference-Time Ordinal Lens Alignment for Multimodal LLMs
Haiming Li | Yingsheng Liu | Jingmin Zhu | Siyuan Yan | Xieji Li | Jiajun Sun | Zongyuan Ge | Zhen yu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Haiming Li | Yingsheng Liu | Jingmin Zhu | Siyuan Yan | Xieji Li | Jiajun Sun | Zongyuan Ge | Zhen yu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Multimodal LLMs apply the language model interface to visual inputs, where ordinal regression tasks such as age estimation, image quality assessment, and disease grading require autoregressive decisions over ordered class labels. We ask whether MLLMs reliably convert internal ordinal evidence into ordered digit-token outputs. Across four ordinal benchmarks and four MLLM backbones, ordinal labels are linearly recoverable from hidden states with Spearman correlation up to 0.938, and a task-designed prompt further sharpens this structure. Yet native digit-token outputs weakly expose it: the unembedding matrix filters the ordinal direction, and the digit-token row space retains below 1.15% across all 16 model-dataset combinations, with a 16 to 77 absolute-point accuracy gap between linear-probe and native outputs. We introduce Ordinal Lens Alignment (OLA), a frozen-backbone inference-time method that trains lightweight Wₛ-anchored lenses on mid-to-deep decoder layers, fuses them into an ordinal distribution, and corrects only digit-token logits at generation. OLA outperforms the SOTA LoRA-tuned OrderChain baseline in most settings while keeping the MLLM frozen, surpasses discriminative ordinal baselines in most cells, and improves over an offline lens in every setting.