Primoz Kocbek

Author directory

2026

Introduction: Biomedical PDFs distribute answer evidence across prose, tables, figures, and cross-document relations, complicating the retrieval of answer-sufficient context. We examine whether graph and multimodal retrieval provide complementary answers and whether direct concatenation exploits this complementarity. Methods: We screen 13 local generators on 205 pharmacokinetics/pharmacodynamics multiple-choice questions and evaluate 11 retrieval conditions over ten runs for five selected models. We quantify questions answered correctly by either or both retrieval routes and compare direct fusion with each standalone route using paired question-level bootstrap intervals. A separate ten-run panel tests answer order, irrelevant context, paraphrasing, generated distractors, and option removal. Results: Multimodal retrieval alone answers 11.5–16.2% of questions correctly that graph retrieval misses, while graph retrieval alone answers 7.9–13.9% that multimodal retrieval misses. Counting a question as correct whenever either route succeeds yields an oracle upper bound of 69.5–77.7%, exceeding direct concatenation by 8.4–13.5 percentage points. Direct fusion improves on the stronger standalone route by at most 0.6 points; paired intervals include zero in all ten model–depth comparisons. In repeated evaluation, Gemma-4-31B with multimodal retrieval achieved the highest accuracy (66.6%, 95% CI 65.4–67.8). Discussion: The central finding is substantial answer-level complementarity that direct concatenation does not reliably exploit. The oracle bound measures retrospective selection, not an implemented routing method. These findings motivate evaluating source-aware routing or reranking to select between complementary retrieval routes.
Automated evaluation of multimodal medical answers is essential for scalable safety assessment, yet it remains difficult to align automatic scores with expert judgment across languages and image modalities. We describe SloCal-Net’s systems for the MEDIQA-EVAL 2026 shared task, framing evaluation as rubric-conditioned multimodal judging: the judge receives the question, image(s), candidate answer, and task-specific criteria, and outputs criterion-level scores and an overall rating. Evidence retrieval was initialized using ChatGPT Deep Research, producing a 25-document clinical corpus used for lightweight retrieval-augmented grounding. On the official leaderboard, our best submission (GPT-5-mini with web search and RAG) achieved Pearson correlations of 0.466 on English and 0.260 on Chinese expert ratings. In post-competition experiments with open-source judges, the best English Pearson reached 0.272 with GLM-4.6V and 0.212 with Qwen3-VL-30B-Thinking, while Chinese correlations were lower, highlighting remaining gaps in multilingual calibration and image–text grounding.