Vision-language pretraining has driven progress in medical image representation learning, but it depends on paired image-text data and can inherit reporting bias from clinical narratives. We study whether language-free predictive pretraining can produce an image encoder that transfers effectively to radiology report generation. RadJEPA is a chest-X-ray adaptation of I-JEPA, pretrained on approximately 840K unlabeled radiographs using latent context-to-target prediction. Our primary contribution is an extensive empirical evaluation of this language-free encoder for report generation: the frozen image encoder is coupled to a trainable two-layer projector and language decoder, and is also substituted into four established vision-language backbones. Across MIMIC-CXR and IU-Xray, RadJEPA matches or exceeds the evaluated image-only and image-text baselines on lexical, entity-relation, and clinical-label metrics. Controlled MIMIC-only comparisons provide evidence that the predictive objective contributes beyond domain-specific pretraining, while broader comparisons also reflect differences in pretraining data, model capacity, and input resolution. Complementary classification and segmentation experiments assess transfer beyond report generation.

The human body is highly interconnected, where a diagnosis in one organ can influence conditions in others. In medical research, graphs (such as Knowledge Graphs and Causal Graphs) have proven useful for capturing these relationships, but constructing them manually with expert input is both costly and time-intensive, especially given the continuous flow of new findings. To address this, we leverage the extraction capabilities of large language models (LLMs) to build the MultiOrgan Diagnosis Knowledge Graph (MOD-KG). MOD-KG contains over 21,200 knowledge triples, derived from both textbooks (~13%) and carefully selected research papers (with an average of 444 citations each). The graph focuses primarily on the heart, lungs, kidneys, liver, pancreas, and brain, which are central to much of today’s multimodal imaging research. The extraction quality of the LLM was benchmarked against baselines over 1000 samples, demonstrating reliability. We will make our dataset public upon acceptance.

Otoscopy is a diagnostic procedure to examine the ear canal and eardrum using an otoscope. It identifies conditions like infections, foreign bodies, eardrum perforations, and ear abnormalities. We propose a novel resource-efficient deep learning and transformer-based framework, Sumotosima (Summarizer for Otoscopic Images), which provides an end-to-end pipeline for classification followed by summarization. Our framework utilizes a combination of triplet and cross-entropy losses. Additionally, we use Knowledge Enhanced Multimodal BART, where the input is fused textual and image embeddings. The objective is to deliver summaries that are well-suited for patients, ensuring clarity and efficiency in understanding otoscopic images. Given the lack of existing datasets, we have curated our own OCASD (Otoscopy Classification And Summary Dataset), which includes 500 images with 5 unique categories, annotated with their class and summaries by otolaryngologists. Sumotosima achieved a result of 98.03%, which is 7.00%, 3.10%, and 3.01% higher than K-Nearest Neighbors, Random Forest, and Support Vector Machines, respectively, in classification tasks. For summarization, Sumotosima outperformed GPT-4o and LLaVA by 88.53% and 107.57% in ROUGE scores, respectively. We have made our code and dataset publicly available at https://github.com/anas2908/Sumotosima