Multimodal long documents are core carriers of professional knowledge, where critical evidence is sparsely distributed across paragraphs and modalities. This easily causes key information omission and cross-modal hallucinations in summarization by multimodal LLMs. These issues stem from attention drift in long-range dependency modeling and gaps in inter-modal alignment. To address this, we introduce MMLDSum-Bench, a high-quality benchmark for multimodal long-document summarization, covering multiple domains, context-length scales, and visual-textual modality distributions. We further propose MMLDSum-LLM, a reproducible two-stage training framework that combines supervised fine-tuning with visual-alignment weighted loss and keyword-aware weighted loss, followed by GRPO with a multi-objective reward (keyword coverage, image-text alignment, ROUGE, and length control). Extensive experiments on MMLDSum-Bench, comparing against leading closed-source and open-source multimodal models under a unified evaluation protocol — including LLM-as-a-judge scoring, atomic-claim precision/recall, image-text alignment (ITA), and ROUGE — demonstrate that our approach significantly improves key-information coverage and cross-modal consistency.

Reward-aware variants of Direct Preference Optimization (DPO), such as ODPO and MODPO, implicitly assume linear marginal utility over the preference signal: identical signal gaps induce identical gradient updates regardless of whether the pair lies in a low- or high-quality region. We propose FreeMU, a lightweight plug-and-play training module that introduces non-linear marginal utility through a pair-aware divided-difference weight applied directly to the preference margin. The same weight formula integrates with three representative DPO variants—ODPO, MODPO, and β-DPO—with each instantiation differing only in the reward source and its normalisation. We first instantiate FreeMU on the two explicit-score variants (Likert labels for ODPO and reward-model outputs for MODPO) and then, for variants without externally annotated scores, on β-DPO using the implicit Bradley–Terry reward induced by the policy and reference models. Across summarization, multi-objective safety alignment, and dialogue, FreeMU consistently improves over its host in LLM-based win rates and task-specific metrics with near-zero training overhead; a controlled ablation confirms that the convex curvature direction is the operative ingredient—reversing the sign to a concave utility removes the gain.