Multimodal Large Language Models (MLLMs) can assign similar confidence to answers that fail for different reasons. We propose HalluPrism, a behavioral diagnostic that re-runs an answer after visual degradation, blank-image replacement, and grounding or relation checks. These targeted probes yield a signature over visual-perturbation sensitivity (V), image-removal confidence retention (L), and grounding/relation-probe instability (A). Across 58K+ examples from four benchmarks and four MLLMs, image-removal confidence retention is most prevalent, while grounding/relation-probe instability better separates failure families. Only 18 of 48 source-target checks are diagonally aligned, so the coordinates should be interpreted jointly rather than as independent causal sources. With the dataset fixed, the joint signature improves failure-family AUROC from 0.634 to 0.769 on HallusionBench and from 0.707 to 0.817 on VizWiz, with smaller gains on POPE and VSR. In pooled XGBoost analysis, AUROC rises from 0.78 with scalar confidence to 0.95 with (V, L, A) and 0.97 when confidence is added. The same signature does not automatically improve correctness ranking. The three tested direct scalarizations can harm it. These results show that multimodal uncertainty should characterize failure structure before it is used to decide whether to abstain or correct.

Document summarization becomes more challenging when summaries must reflect a user’s subjective interests in addition to document salience. SOTA Large Language Models (LLMs) show strong in-context summarization capabilities. Prior works report that simply prepending long and dynamically evolving user histories leads to unstable, inconsistent personalization. To address this, we introduce PerDucer, a personalization inducer for frozen language models. Given a user interaction sequence (trajectory) and a query document, PerDucer first predicts the next likely preference signal. It then maps the latent signal to a small set of personalized keyphrases for the query document. These keyphrases serve as the control cues that steer the frozen summarizers (both LLMs and SLMs) towards user-aligned summaries. Across the PENS and OpenAI-Reddit benchmarks, PerDucer-boosted LLMs consistently outperform the strongest history-prompting baselines, yielding an average +0.18 improvement across personalization metrics (PerSEval in our case). Two PerDucer-augmented SLMs approach the top-performing evaluated LLM, with SmolLM2-1.7B reaching 97% of the best-performing DeepSeek-R1-14B score. These results indicate that short keyphrase cues can induce personalization in frozen summarizers without modifying their parameters.

Text summarization models are evaluated in terms of their accuracy and quality using various measures such as ROUGE, BLEU, METEOR, BERTScore, PYRAMID, readability, and several other recently proposed ones. The central objective of all accuracy measures is to evaluate the model’s ability to capture saliency accurately. Since saliency is subjective w.r.t the readers’ preferences, there cannot be a fit-all summary for a given document. This means that in many use-cases, summarization models need to be personalized w.r.t user-profiles. However, to our knowledge, there is no measure to evaluate the degree-of-personalization of a summarization model. In this paper, we first establish that existing accuracy measures cannot evaluate the degree of personalization of any summarization model, and then propose a novel measure, called EGISES, for automatically computing the same. Using the PENS dataset released by Microsoft Research, we analyze the degree of personalization of ten different state-of-the-art summarization models (both extractive and abstractive), five of which are explicitly trained for personalized summarization, and the remaining are appropriated to exhibit personalization. We conclude by proposing a generalized accuracy measure, called P-Accuracy, for designing accuracy measures that should also take personalization into account and demonstrate the robustness and reliability of the measure through meta-evaluation.