Thomas Frauenfelder
Author directory2026
Universal Boosts, Specific Suppressors: Sparse Autoencoder Steering of Medical Vision-Language Models
Farhad Nooralahzadeh | Benjamin Gundersen | Nicolas Deperrois | Hidetoshi Matsuo | Mizuho Nishio | Thomas Frauenfelder | Ahmed Allam | Christian Bluethgen | Michael Moor | Michael Krauthammer
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Farhad Nooralahzadeh | Benjamin Gundersen | Nicolas Deperrois | Hidetoshi Matsuo | Mizuho Nishio | Thomas Frauenfelder | Ahmed Allam | Christian Bluethgen | Michael Moor | Michael Krauthammer
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Medical vision-language models (VLMs) commit clinical errors when generating chest X-ray reports: they fabricate findings that are not present in the image, miss important ones, or locate them incorrectly. We steer the error composition of generated reports without weight updates by decoding-time residual steering on a per-token sparse autoencoder (SAE) basis: Top-K SAEs on late layers, per-error-type causal feature screening against clinical errors, then combined suppress/boost intervention at inference time. On the MIMIC-CXR test split, our inference-only method improves the quality of generated reports for three radiology VLMs (RadVLM, LLaVA-Rad, and CheXOne), with relative improvements of +5.4%, +7.2%, and +17.0% in the clinical composite metric, and statistically significant GREEN gains on all backbones. The gain is an explicit, controllable trade-off rather than uniform error reduction: steering shifts models toward more complete reporting, substantially fewer missing findings, at the cost of some additional false findings, and we analyse and quantify this redistribution per error type. A cross-model feature alignment shows that the quality-promoting (boost) directions have functionally similar error-type profiles across architectures, whereas hallucination-linkedm (suppress) directions are model-specific. Census-motivated steering should therefore treat suppression per-backbone, rather than sharing a universal suppress list. The same recipe applied zero-shot to IU-Xray (GREEN +7.7% rel., on RadVLM) suggests that the identified features are properties of the model rather than of the training corpus. The causal feature sets and an interactive feature dashboard are shared.
2021
Progressive Transformer-Based Generation of Radiology Reports
Farhad Nooralahzadeh | Nicolas Perez Gonzalez | Thomas Frauenfelder | Koji Fujimoto | Michael Krauthammer
Findings of the Association for Computational Linguistics: EMNLP 2021
Farhad Nooralahzadeh | Nicolas Perez Gonzalez | Thomas Frauenfelder | Koji Fujimoto | Michael Krauthammer
Findings of the Association for Computational Linguistics: EMNLP 2021
Inspired by Curriculum Learning, we propose a consecutive (i.e., image-to-text-to-text) generation framework where we divide the problem of radiology report generation into two steps. Contrary to generating the full radiology report from the image at once, the model generates global concepts from the image in the first step and then reforms them into finer and coherent texts using transformer-based architecture. We follow the transformer-based sequence-to-sequence paradigm at each step. We improve upon the state-of-the-art on two benchmark datasets.