Debshree Chowdhury
Author directory2026
Modality-Specific and Relation-Aware Modeling for Scientific Figure Linking
Bodhisatta Maiti | Debshree Chowdhury
Proceedings of the 4th Workshop on Artificial Intelligence for Scientific Publications
Bodhisatta Maiti | Debshree Chowdhury
Proceedings of the 4th Workshop on Artificial Intelligence for Scientific Publications
We present a modality-specific system for the AstroCLIMB shared task, which predicts relations among astronomical figures and captions. Caption–caption, image–image, and image–caption pairs use different combinations of frozen pretrained representations and supervised classifiers. Image–caption prediction first applies a Qwen3-VL 4B/8B exact correspondence gate before classifying the remaining paper relations. Subsequent improvements add astronomy-domain sentence embeddings to the caption branch and a learned visual-projection specialist to the image branch. Across eight Kaggle submissions, macro-F1 increases from 0.3641 to 0.6096. Adding astronomy-specific caption features improves five-fold pooled out-of-fold (OOF) macro-F1 from 0.6071 to 0.6388; the image specialist offers a smaller incremental test-set gain. The direct citation relation, related_papers, remains the most difficult. Results support selecting representations and decision mechanisms according to modality, while highlighting the limitations of recovering citation links from isolated content.