Automated radiology report generation is expanding across imaging modalities and anatomical regions, yet its evaluation remains misaligned with diagnostic priorities. Existing metrics either rely on surface-level text similarity or require slow, costly generative inference. This limits their ability to detect clinically significant errors, such as hallucinated or omitted abnormal findings, and prevents their use as efficient reward signals during training. We introduce POLAR (POlarity-Aware Long-context Anatomy-conditioned Regression), an encoder-based evaluation metric built on ModernBERT. POLAR combines long-context inputs for multi-organ reports, anatomy conditioning for targeted body-region evaluation, and polarity-aware regression that explicitly distinguishes normal from abnormal findings, yielding a more clinically meaningful assessment than prior metrics. Across real-world, radiologist-annotated CXR benchmarks and expert preference datasets, POLAR achieves competitive alignment with radiologist judgments while running 11x faster than generative evaluators, making it practical for large-scale benchmarking and as a training-time reward signal. We publicly release POLAR and the POLAR-Synth dataset, along with a fully open-source, privacy-preserving pipeline for data generation, training, and inference, enabling the metric to be extended to new modalities and anatomies.