Medical Image Quality Assessment (IQA) serves as a critical safety gate for clinical AI, yet existing approaches rely on scalar metrics that fail to capture the descriptive, expert-like evaluation process central to clinical practice. We introduce MedQ-Bench, a comprehensive benchmark for language-based medical IQA evaluation with MLLMs, establishing a perception–description paradigm across 5 imaging modalities and over 40 quality attributes. The benchmark comprises two complementary tasks: MedQ-Perception probes low-level visual attribute discrimination via 2,600 expert-annotated perceptual queries; MedQ-Description covers both no-reference and comparison description sub-tasks with 708 expert-annotated samples, evaluated through a four-dimensional judging protocol. Comprehensive evaluation of 23 state-of-the-art MLLMs reveals that current models exhibit only preliminary and unstable capabilities, and highlights the urgent need for targeted optimization of MLLMs in medical IQA. Project Dataset: https://github.com/liujiyaoFDU/MedQ-Bench

Large language model (LLM)-based judges are increasingly used to scale the evaluation and optimization of open-ended medical dialogue systems, making their reliability critical in this high-stakes setting. However, existing medical benchmarks mainly assess their medical knowledge, while general judge benchmarks lack clinically grounded consultation criteria. To address this gap, we introduce Med-GRADE, a benchmark for evaluating LLMs as medical dialogue judges. It contains 2,076 real-world patient–LLM dialogues annotated by 56 physicians with a 23-item rubric adapted from Mini-CEX, covering Medical Interviewing (MI), Humanistic Care (HC), and Diagnosis and Treatment Management (DTM). Experiments on 17 LLMs show that current LLM judges achieve only moderate agreement, with the best model reaching 64.89% Macro-F1 and 68.28% Hamming Accuracy. Besides, more complex prompts may even harm performance: CoT disrupts parsing, while few-shot and self-refine introduce item-level instability. Data and code are available at https://github.com/jyd777/Med-GRADE for reproducibility.