Byunggill Joe
Author directory2026
Robust Multimodal Sentiment Analysis with Incomplete Modalities via Semantic-aware Completeness based Reconstruction
Han-Jun Choi | Byunggill Joe | Saim Shin | Jin Yea Jang
Findings of the Association for Computational Linguistics: EMNLP 2026
Han-Jun Choi | Byunggill Joe | Saim Shin | Jin Yea Jang
Findings of the Association for Computational Linguistics: EMNLP 2026
Recent multimodal sentiment analysis studies increasingly adopt text-centric fusion approaches to exploit the rich sentiment information inherent in the textual modality. However, these approaches often suffer from performance degradation during inference due to partially missing or noisy data in real-world scenarios, especially when sentiment-related cues are missing. To address this issue, we introduce a new completeness estimation approach that quantifies the degree of sentiment-relevant information preserved in incomplete data to guide the reconstruction of missing semantics. Furthermore, we propose a training strategy that stabilizes multi-task learning while jointly optimizing sentiment prediction and completeness estimation. Extensive experiments and in-depth analyses on three benchmark datasets demonstrate that the proposed approach enables more accurate semantic reconstruction, leading to more precise sentiment prediction.
KoMMERCE: A Korean Multimodal and Multitask Evaluation Benchmark for Reliable Conversational E-commerce Agents
Byunggill Joe | San Kim | Minyoung Jung | Dong Hyok Shin | Yu-Jeong Sung | Saim Shin
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Byunggill Joe | San Kim | Minyoung Jung | Dong Hyok Shin | Yu-Jeong Sung | Saim Shin
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
E-commerce conversations are increasingly multimodal, yet end-to-end response scores cannot localize which dialogue competence failed, and no public benchmark evaluates finegrained multimodal dialogue understanding for Korean commercial agents. We present KoMMERCE, a Korean multimodal fashion-domain benchmark that coordinates nine fine-grained dialogue understanding tasks on the same conversations: sentiment, intent, and strategy classification;image and text knowledge retrieval;dialogue state tracking; span detection; coreference resolution; and the first formulation of cross-modal dialogue relation extraction. The corpus contains 15,293 dyadic role-play dialogues under two-stage quality control. Baselines on six frontier multimodal LLMs reveal task-conditional patterns that should inform deployment choices: no model wins all nine tasks, and context length affects accuracy differently from one task to another.