Although LLM-as-a-Judge approaches have been successful in several domains, recent studies show that LLMs still struggle with Computational Argumentation tasks, particularly Argument Quality Assessment. Motivated by recent results in Information Retrieval showing that scoring granularity can affect the effectiveness of LLM-based document rankers, this paper investigates whether the same effect holds in Argument Quality Assessment scenarios.

Our results demonstrate that, in several cases, more granular scoring ranges yield substantially higher agreement after mapping predictions back to the original scale. This contrasts with how humans evaluate, where reducing the granularity of possible scores can foster clearer, more consistent decisions. Additionally, our error analysis reveals a recurring limitation: LLMs tend to underestimate arguments labeled by human experts as high quality, frequently assigning them medium scores instead. This suggests that, despite the gains obtained through prompt and scale design, current LLMs remain poorly calibrated at the upper end of the argument quality scale.