Hui Ji
Author directory2026
FigEx2: Visual-Conditioned Panel Detection and Captioning for Scientific Compound Figures
Jifeng Song | Arun Das | Pan Wang | Hui Ji | Kun Zhao | Yufei Huang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jifeng Song | Arun Das | Pan Wang | Hui Ji | Kun Zhao | Yufei Huang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Scientific compound figures combine multiple labeled panels into a single image, and downstream pretraining and retrieval require panel-aligned visual-text pairs. However, in a PubMed Central (PMC)-scale crawl of 346,567 compound figures, 16.3% have no caption and are discarded by existing caption-decomposition pipelines. We propose FigEx2, a visual-conditioned framework that takes only a compound figure as input and jointly produces labeled panel boxes and panel-wise captions. FigEx2 introduces an Entity-Attention Kullback-Leibler (KL) regularizer that aligns the detector’s cross-attention with scientific entities annotated for each panel, providing a stable conditioning signal that also improves localization, and applies Group Relative Policy Optimization (GRPO) with a panel-level Entity-F1 reward to optimize scientific faithfulness. We curate BioSci-Fig-Cap for in-domain supervision and contribute physics and chemistry test suites for cross-disciplinary evaluation. FigEx2 achieves 0.751 mAP@0.5:0.95 on BioSci-Fig-Cap, and outperforms Qwen3-VL-8B by 6.80 Entity-F1 on MedICaT for captioning. It also transfers zero-shot to out-of-distribution domains.
2024
RAG-RLRC-LaySum at BioLaySumm: Integrating Retrieval-Augmented Generation and Readability Control for Layman Summarization of Biomedical Texts
Yuelyu Ji | Zhuochun Li | Rui Meng | Sonish Sivarajkumar | Yanshan Wang | Zeshui Yu | Hui Ji | Yushui Han | Hanyu Zeng | Daqing He
Proceedings of the 23rd Workshop on Biomedical Natural Language Processing
Yuelyu Ji | Zhuochun Li | Rui Meng | Sonish Sivarajkumar | Yanshan Wang | Zeshui Yu | Hui Ji | Yushui Han | Hanyu Zeng | Daqing He
Proceedings of the 23rd Workshop on Biomedical Natural Language Processing
This paper introduces the RAG-RLRC-LaySum framework, designed to make complex biomedical research accessible to laymen through advanced Natural Language Processing (NLP) techniques. Our innovative Retrieval Augmentation Generation (RAG) solution, enhanced by a reranking method, utilizes multiple knowledge sources to ensure the precision and pertinence of lay summaries. Additionally, our Reinforcement Learning for Readability Control (RLRC) strategy improves readability, making scientific content comprehensible to non-specialists. Evaluations using the publicly accessible PLOS and eLife datasets show that our methods surpass Plain Gemini model, demonstrating a 20% increase in readability scores, a 15% improvement in ROUGE-2 relevance scores, and a 10% enhancement in factual accuracy. The RAG-RLRC-LaySum framework effectively democratizes scientific knowledge, enhancing public engagement with biomedical discoveries.
Search