LLM-as-judge frameworks are widely used for subjective NLP evaluation, yet their reliability on social-cognition tasks remains unclear. We investigate whether Theory-of-Mind (ToM) scaffolding can help LLM judges evaluate persuasion on Reddit ChangeMyView (CMV). We test four judges (GPT-5, GPT-4.1, Claude Sonnet 4.6, Qwen3-30B-A3B-Instruct-2507) under three conditions that share the same ToM annotations but vary how they are presented: No-ToM, Full-ToM (flat), and our proposed PAToM (Persuasion-Aware ToM), which structures reasoning across five sequential belief-stack turns. Our results show that better reasoning does not lead to better verdicts. Although PAToM improves reasoning coherence by 1.6x, semantic coverage by 1.1-1.8x, and our proposed process metric PRISM by 2-3x, all four judges still struggle to reliably distinguish persuasive comments from high-quality non-persuasive ones. Process-level analysis shows that reasoning quality and verdict accuracy are empirically decoupled under this scaffolding, an open challenge for prompt-based approaches to persuasion judging. We release prompts, ToM annotations, the PRISM metric suite, and a 1000-item benchmark.

In the midst of widespread misinformation and disinformation through social media and the proliferation of AI-generated texts, it has become increasingly difficult for people to validate and trust information they encounter. Many fact-checking approaches and tools have been developed, but they often lack appropriate explainability or granularity to be useful in various contexts. A text validation method that is easy to use, accessible, and can perform fine-grained evidence attribution has become crucial. More importantly, building user trust in such a method requires presenting the rationale behind each prediction, as research shows this significantly influences people’s belief in automated systems. Localizing and bringing users’ attention to the specific problematic content is also paramount, instead of providing simple blanket labels. In this paper, we present ClaimVer, a human-centric framework tailored to meet users’ informational and verification needs by generating rich annotations and thereby reducing cognitive load. Designed to deliver comprehensive evaluations of texts, it highlights each claim, verifies it against a trusted knowledge graph (KG), presents the evidence, and provides succinct, clear explanations for each claim prediction. Finally, our framework introduces an attribution score, enhancing applicability across a wide range of downstream tasks.