Jaseel Muhammad Kaithakkodan
Author directory2026
Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation
Mohammed Irfan Kurpath | Jaseel Muhammad Kaithakkodan | Sahal Shaji Mullappilly | Ivan Laptev | Hisham Cholakkal
Findings of the Association for Computational Linguistics: EMNLP 2026
Mohammed Irfan Kurpath | Jaseel Muhammad Kaithakkodan | Sahal Shaji Mullappilly | Ivan Laptev | Hisham Cholakkal
Findings of the Association for Computational Linguistics: EMNLP 2026
Extending a text embedding model to new modalities typically degrades text retrieval quality, and existing omni-modal embedders compensate with multi-billion parameters. We present Omni-Embed-Mini, a 0.9B-parameter model that maps text, speech, audio, images, video, and visually-rich documents into a single shared cosine space without updating any text-side parameter. Our key insight is that the teacher signal requires no separate model: each media sample is paired with a dense cascaded caption, and the teacher target is simply the frozen backbone’s own embedding of that caption. Because teacher and student share the same backbone weights, they inhabit byte-identical geometry, and lightweight projectors plus phased LoRA adapters on the modality encoders suffice for alignment. Training combines a Matryoshka SigLIP contrastive loss with an online hybrid hard-negative miner whose negatives sharpen as the encoder improves. The recipe scales unchanged to a 2.3B variant by swapping in a native vision-language backbone. Omni-Embed-Mini-0.9B keeps its text weights bit-identical to the backbone, so training cannot regress text retrieval (49.57 nDCG@10 on MTEB-v2 BEIR-8), while extending it to five additional modalities, and is ~2.7× to 9.5× smaller than every open omni embedder we compare against. The 2.3B variant is competitive with closed API embedders, edging ahead of gemini-embedding-2 on the overall-modality average. Models, code, dataset, and evaluation harness are available on our project page: https://omniembed.cvmbzuai.com
MediX-R1: Open Ended Medical Reinforcement Learning
Sahal Shaji Mullappilly | Mohammed Irfan Kurpath | Jaseel Muhammad Kaithakkodan | Omair Mohamed | Mohamed Zidan | Fahad Shahbaz Khan | Salman Khan | Rao Muhammad Anwer | Hisham Cholakkal
Findings of the Association for Computational Linguistics: EMNLP 2026
Sahal Shaji Mullappilly | Mohammed Irfan Kurpath | Jaseel Muhammad Kaithakkodan | Omair Mohamed | Mohamed Zidan | Fahad Shahbaz Khan | Salman Khan | Rao Muhammad Anwer | Hisham Cholakkal
Findings of the Association for Computational Linguistics: EMNLP 2026
In this work, we introduce MediX-R1, an open-ended Reinforcement Learning (RL) framework for medical vision language models (VLMs) that enables free-form answers beyond multiple-choice formats. MediX-R1 fine-tunes a baseline vision-language backbone with Group Based RL and a composite reward tailored for medical reasoning: an LLM-based accuracy reward that judges semantic correctness with a strict YES/NO decision, a medical embedding-based semantic reward to capture paraphrases and terminology variants, and lightweight format and modality rewards that enforce interpretable reasoning and modality recognition. This multi-signal design provides stable, informative feedback for open-ended outputs where traditional verifiable or MCQ-only rewards fall short. To evaluate performance, we employ a unified evaluation framework for both text-only and multimodal (image+text) tasks that uses a Reference-based LLM-as-judge in place of brittle string-overlap metrics, capturing semantic correctness, clinical accuracy, and contextual alignment. Our results show that open-ended RL with comprehensive reward signals can effectively strengthen medical reasoning in multimodal models. Despite using only ~51K instruction examples, MediX-R1 achieves strong performance across standard medical LLM (text-only) and VLM (image + text) benchmarks, outperforming existing open-weight medical VLMs, with large gains particularly on open-ended clinical tasks. These improvements remain consistent across multiple RL algorithms and VLM backbones and are further supported by blinded evaluations with medical experts. Our trained models, curated datasets and source code are available at https://medix.cvmbzuai.com