Thomas Bonald

Author directory

2026

Reliable evaluation of factual recall in large language models (LLMs) requires ground truth that is broad, clean, and complete. Yet existing benchmarks often rely on noisy extraction, narrow domains, or incomplete knowledge bases that penalize models for generating correct facts absent from the reference. We introduce **MUSART**, a benchmark of 112 596 factual triples over 31 domains, automatically constructed by pairing Wikipedia Good Article categories with Wikidata and selecting domain-salient relations via Personalized PageRank. To overcome knowledge-base incompleteness, we contribute an automated, character-grounded augmentation pipeline that mines verbatim facts from source articles. Evaluating twenty-one open-weight LLMs against both reference regimes reveals that (i) completeness correction systematically raises precision and lowers recall across all models, especially on multi-valued relations; and (ii) factual recall varies widely across subject domains, demonstrating why single-domain evaluation confounds model capability with subject matter. All code, data, and evaluation scripts are publicly released at https://github.com/Rajjaa/MUSART.

2024

Hierarchical text classification (HTC) is the task of assigning labels to a text within a structured space organized as a hierarchy. Recent works treat HTC as a conventional multilabel classification problem, therefore evaluating it as such. We instead propose to evaluate models based on specifically designed hierarchical metrics and we demonstrate the intricacy of metric choice and prediction inference method. We introduce a new challenging dataset and we evaluate fairly, recent sophisticated models, comparing them with a range of simple but strong baselines, including a new theoretically motivated loss. Finally, we show that those baselines are very often competitive with the latest models. This highlights the importance of carefully considering the evaluation methodology when proposing new methods for HTC