Jihwan Kim
Author directory2026
Can Omni-models Imagine the Future? A Diagnostic Benchmark for Multimodal World Modeling
Junyeob Kim | Jihwan Kim | Minha Jhang | Sungwook Jeon | Hwiyeol Jo | Jeonghoon Kim | Sang-goo Lee | Joosung Lee
Findings of the Association for Computational Linguistics: EMNLP 2026
Junyeob Kim | Jihwan Kim | Minha Jhang | Sungwook Jeon | Hwiyeol Jo | Jeonghoon Kim | Sang-goo Lee | Joosung Lee
Findings of the Association for Computational Linguistics: EMNLP 2026
Recent progress in Omni-models has yielded strong performance on benchmarks focused on static observations. However, their ability to maintain spatiotemporal continuity when predicting coherent future states from previous context remains underexplored. We address this by formulating Next Scene Prediction (NSP) as a diagnostic task for assessing multimodal near-future state prediction, requiring models to anticipate the immediate future from a given context. We introduce OmniNSP, a benchmark evaluating causal and cross-modal coherence across physical, logical, and social dynamics. Unlike prior tasks that often rely on textual or discriminative proxies, OmniNSP asks models to express predicted future states through text-mediated visual and speech synthesis. Our analysis shows that current Omni-models can produce plausible textual forecasts while still failing to preserve future-state consistency in generated outputs. A frozen-generator control substantially narrows cross-model visual-consistency gaps, attributing much of the original system-level difference to model-specific realization stacks rather than textual foresight alone. A modality-input ablation further shows that audio provides complementary predictive cues beyond vision, with the largest gains for logical transitions and contexts with informative speech. These results position OmniNSP as a diagnostic of both near-future prediction and the preservation of predicted state through multimodal realization.
Retrieve Only Relevant Tables Whether Few or Many: Adaptive Table Retrieval Method
Taehee Kim | Seungbin Yang | Jihwan Kim | Jaegul Choo
Findings of the Association for Computational Linguistics: ACL 2026
Taehee Kim | Seungbin Yang | Jihwan Kim | Jaegul Choo
Findings of the Association for Computational Linguistics: ACL 2026
Retrieving relevant tables from extensive databases for a given natural language query is essential for accurately answering questions in tasks such as text-to-SQL. Existing table retrieval approaches select a pre-determined set of k tables with the highest similarity to the query. However, the number of required tables varies across queries and cannot be known in advance. Enforcing a fixed number of retrieved tables regardless of the query may either retrieve an undersized set, failing to obtain all necessary evidence, or retrieve an oversized pool, including irrelevant tables. To address this issue, we propose an adaptive table retrieval method that adjusts the number of tables retrieved according to the requirements of each query. Specifically, we utilize an adaptive thresholding mechanism to selectively retrieve tables and integrate a sliding-window reranking algorithm to efficiently process a large table corpus. Extensive experiments on Spider, BIRD, and Spider 2.0 demonstrate that our method effectively addresses the limitations of the top-k retrieval strategy, improving performance in retrieval and downstream tasks. Our code and data are available at https://anonymous.4open.science/r/Adaptive-Table-Retrieval.
From Unverifiable Domain Text to Cross-Lingual RLVR via Code-Switched Task Synthesis
Cheonbok Park | Jihwan Kim | Shinyoung Joo | Joosung Lee | Hwiyeol Jo | Jeonghoon Kim | Jaegul Choo
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Cheonbok Park | Jihwan Kim | Shinyoung Joo | Joosung Lee | Hwiyeol Jo | Jeonghoon Kim | Jaegul Choo
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Reinforcement Learning with Verifiable Rewards (RLVR) can improve reasoning in large language models, but specialized non-English domains expose a data mismatch. Raw English domain text is often available, while target-language RLVR tasks with verifiable rewards are scarce. We introduce code-switched task synthesis for cross-lingual RLVR, a data-centric approach that derives target-language and code-switched RLVR task variants from otherwise unverifiable English domain text. Using fill-in-the-middle multiple-choice QA (FIM-MCQA) as a seed task format, we synthesize automatically verifiable tasks from domain passages and reshape them into cross-lingual variants. These variants preserve the same evidence-answer relation but change how English and target-language evidence are presented during RLVR. On Korean medical benchmarks and cybersecurity benchmarks across four target languages, code-switched training outperforms matched monolingual and bilingual-mixture baselines on average target-language performance while maintaining comparable English performance. Additional analyses provide evidence of better target-language input understanding and stronger cross-lingual alignment in both model outputs and internal representations. These results suggest that code-switched training helps target-language inputs access domain reasoning learned more strongly in English.
Sommelier: Scalable Open Multi-turn Audio Pre-processing for Full-duplex Speech Language Models
Kyudan Jung | Jihwan Kim | Soyoon Kim | Jeonghoon Kim | Jaegul Choo | Cheonbok Park
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)
Kyudan Jung | Jihwan Kim | Soyoon Kim | Jeonghoon Kim | Jaegul Choo | Cheonbok Park
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)
As the paradigm of AI shifts from text-based LLMs to Speech Language Models (SLMs), there is a growing demand for full-duplex systems capable of real-time, natural human-computer interaction.However, the development of such models is constrained by the scarcity of high-quality, multi-speaker conversational data, as existing large-scale resources are predominantly single-speaker or limited in volume.Addressing the complex dynamics of natural dialogue, such as overlapping and back-channeling remains a challenge, with standard processing pipelines suffering from diarization errors and ASR hallucinations.To bridge this gap, we present a robust and scalable open-source data processing pipeline designed for full-duplex model.Our code and project page are publicly available at https://anonymous-2001-j.github.io/sommelier.github.io/.