Abstention is the intentional refusal of models to provide an answer. Adverse drug reactions (ADRs), which range from mild to life-threatening, are often underreported by patients in hospitals and to doctors because patients assume these are unavoidable. With the growing adoption of LLMs and vision language models (VLMs) in healthcare, patients are increasingly reaching out to these models for solutions. In such high-risk scenarios, incorrect or overconfident responses can cause unnecessary anxiety, lead patients to discontinue essential medications, or encourage inappropriate self-medication. In this work, we propose the MAD-MED, a Model Abstention approach for Drug Adverse Reaction for MEDical-VLMs. It is a training strategy to encourage models to abstain when answering would contradict the available input, exceed the model’s knowledge, or violate model ethics. In addition, we release an augmented Adverse Drug Reaction ABStention (ADR-ABS) dataset comprising image-answerable, text-answerable, and unanswerable questions to support systematic evaluation of abstention behavior. Quantitative and human evaluations confirm that our proposed MAD-MED approach improved the abstention capabilities of models considerably while keeping the generative capabilities intact in cases where the model is expected to answer.