Hongyu Jing
2025
Enhancing Multimodal Named Entity Recognition through Adaptive Mixup Image Augmentation
Bo Xu
|
Haiqi Jiang
|
Jie Wei
|
Hongyu Jing
|
Ming Du
|
Hui Song
|
Hongya Wang
|
Yanghua Xiao
Proceedings of the 31st International Conference on Computational Linguistics
Multimodal named entity recognition (MNER) extends traditional named entity recognition (NER) by integrating visual and textual information. However, current methods still face significant challenges due to the text-image mismatch problem. Recent advancements in text-to-image synthesis provide promising solutions, as synthesized images can introduce additional visual context to enhance MNER model performance. To fully leverage the benefits of both original and synthesized images, we propose an adaptive mixup image augmentation method. This method generates augmented images by determining the mixing ratio based on the matching score between the text and image, utilizing a triplet loss-based Gaussian Mixture Model (TL-GMM). Our approach is highly adaptable and can be seamlessly integrated into existing MNER models. Extensive experiments demonstrate consistent performance improvements, and detailed ablation studies and case studies confirm the effectiveness of our method.
Boosting Text-to-SQL through Multi-grained Error Identification
Bo Xu
|
Shufei Li
|
Hongyu Jing
|
Ming Du
|
Hui Song
|
Hongya Wang
|
Yanghua Xiao
Proceedings of the 31st International Conference on Computational Linguistics
Text-to-SQL is a technology that converts natural language questions into executable SQL queries, allowing users to query and manage relational databases more easily. In recent years, large language models have significantly advanced the development of text-to-SQL. However, existing methods often overlook validation of the generated results during the SQL generation process. Current error identification methods are mainly divided into self-correction approaches based on large models and feedback methods based on SQL execution, both of which have limitations. We categorize SQL errors into three main types: system errors, skeleton errors, and value errors, and propose a multi-grained error identification method. Experimental results demonstrate that this method can be integrated as a plugin into various methods, providing effective error identification and correction capabilities.
Search
Fix data
Co-authors
- Ming Du 2
- Hui Song (宋晖) 2
- Hongya Wang 2
- Yanghua Xiao 2
- Bo Xu (徐波, 徐博) 2
- show all...