Danping Yan
Author directory2026
Beyond Warmth: Risk-Tiered Auditing of Credibility in LLM Companion Responses
Danping Yan
Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026)
Danping Yan
Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026)
Large language models (LLMs) are increasingly used as conversational companions, where warm and supportive responses may still become poorly scoped as guidance under uncertainty or risk. Existing evaluations often emphasize affective alignment, helpfulness, persona consistency, or overt safety violations, leaving less attention to failures where caring language becomes action-relevant but insufficiently bounded. We introduce a risk-tiered audit framework for credibility under risk in supportive companion responses, evaluating whether responses remain responsible, bounded, and behaviorally appropriate when interpreted as guidance. The framework combines fixed scenario-level risk tiers, five human-facing diagnostic credibility dimensions, and response-level failure tags to identify when warm support becomes poorly scoped guidance. Risk tiers indicate where mis-scoped guidance would be more consequential, while credibility scores and failure tags characterize response behavior in those contexts. Across a heterogeneous response pool, we find a recurring tiered failure pattern: HIGH-tier scenarios selectively expose responsibility- and behavior-facing failures even when tone and identity remain comparatively stable. These failures are dominated by overconfident guidance, where models discourage qualified escalation or commit too confidently to action under uncertainty. Cross-judge diagnostics, human validation, and portability analyses support a conservative interpretation of this credibility signal. Overall, the results suggest that supportive dialogue systems should be evaluated not only for warmth, but also for whether support remains responsible, bounded, and well-scoped when guidance carries higher stakes.