Benjamin Gundersen
Author directory2026
Universal Boosts, Specific Suppressors: Sparse Autoencoder Steering of Medical Vision-Language Models
Farhad Nooralahzadeh | Benjamin Gundersen | Nicolas Deperrois | Hidetoshi Matsuo | Mizuho Nishio | Thomas Frauenfelder | Ahmed Allam | Christian Bluethgen | Michael Moor | Michael Krauthammer
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Farhad Nooralahzadeh | Benjamin Gundersen | Nicolas Deperrois | Hidetoshi Matsuo | Mizuho Nishio | Thomas Frauenfelder | Ahmed Allam | Christian Bluethgen | Michael Moor | Michael Krauthammer
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Medical vision-language models (VLMs) commit clinical errors when generating chest X-ray reports: they fabricate findings that are not present in the image, miss important ones, or locate them incorrectly. We steer the error composition of generated reports without weight updates by decoding-time residual steering on a per-token sparse autoencoder (SAE) basis: Top-K SAEs on late layers, per-error-type causal feature screening against clinical errors, then combined suppress/boost intervention at inference time. On the MIMIC-CXR test split, our inference-only method improves the quality of generated reports for three radiology VLMs (RadVLM, LLaVA-Rad, and CheXOne), with relative improvements of +5.4%, +7.2%, and +17.0% in the clinical composite metric, and statistically significant GREEN gains on all backbones. The gain is an explicit, controllable trade-off rather than uniform error reduction: steering shifts models toward more complete reporting, substantially fewer missing findings, at the cost of some additional false findings, and we analyse and quantify this redistribution per error type. A cross-model feature alignment shows that the quality-promoting (boost) directions have functionally similar error-type profiles across architectures, whereas hallucination-linkedm (suppress) directions are model-specific. Census-motivated steering should therefore treat suppression per-backbone, rather than sharing a universal suppress list. The same recipe applied zero-shot to IU-Xray (GREEN +7.7% rel., on RadVLM) suggests that the identified features are properties of the model rather than of the training corpus. The causal feature sets and an interactive feature dashboard are shared.