Hong Jia
Author directory2026
MANGO: Manga Active Narrative Grounding Optimization
Hao Qiu | Junyan Wang | Zheyuan Liu | Lei Fan | Hong Jia | Lianbo Guo | Zhulin Tao
Findings of the Association for Computational Linguistics: EMNLP 2026
Hao Qiu | Junyan Wang | Zheyuan Liu | Lei Fan | Hong Jia | Lianbo Guo | Zhulin Tao
Findings of the Association for Computational Linguistics: EMNLP 2026
Manga visual question answering requires models to answer questions over panel-based visual narratives, where relevant evidence is distributed across ordered panels, embedded text, recurring characters, and implicit event transitions. This structure makes passive page encoding insufficient, as the model must identify which panels to inspect, what clues to retain, and when the accumulated evidence is sufficient for answering. We propose MANGO (Manga Active Narrative Grounding Optimization), an unsupervised framework for active manga visual question answering. MANGO introduces Active Narrative Sketching (ANS), which iteratively selects panels, extracts concise grounded clues, and decides when to stop, forming a compact question-directed evidence sketch before answer generation. To optimize this behavior without human-annotated answers or rationale paths, MANGO samples multiple ANS rollouts and applies group-relative training with two rewards: answer preference from listwise self-ranking and path consistency from stable ordered panel trajectories. The combined reward is optimized with group-relative policy training, encouraging the model to improve both final answers and the panel-level evidence paths that support them. Experiments on standard manga understanding benchmarks show that MANGO achieves state-of-the-art performance across different settings.
Why Can’t They Remember? Uncovering Representation and Retrieval Bottlenecks in Multi-Turn Acoustic Memory
Yang Xiao | Siyi Wang | Han Yin | Hong Jia | Vidhyasaharan Sethu | Eun-Jung Holden | Ting Dang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yang Xiao | Siyi Wang | Han Yin | Hong Jia | Vidhyasaharan Sethu | Eun-Jung Holden | Ting Dang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large audio language models (LALMs) process both speech and environmental acoustic cues, yet struggle to retain non-speech information across multi-turn interactions. The performance gap between semantic (speech) and acoustic (non-speech) understanding remains poorly understood, and the underlying mechanisms of representation and retrieval are still unclear. This work introduces EnvMem, a controlled multi-turn benchmark designed to study this gap and identify the root causes of failures at the representation (i.e., latent embeddings) and retrieval levels (i.e., attention allocation). We further conduct post-hoc interventions to probe representational structure and attention dynamics. Our results reveal representational trajectory drift as the key failure mode, while showing that attention allocation tested here does not consistently explain the degradation. Overall, we provide a systematic framework for analyzing and improving non-linguistic memory in long-context LALMs, shedding light on future data and training design for robust acoustic memory modeling.