Rajaa El Hamdani

Author directory

2026

Reliable evaluation of factual recall in large language models (LLMs) requires ground truth that is broad, clean, and complete. Yet existing benchmarks often rely on noisy extraction, narrow domains, or incomplete knowledge bases that penalize models for generating correct facts absent from the reference. We introduce MUSART, a benchmark of 112 596 factual triples over 31 domains, automatically constructed by pairing Wikipedia Good Article categories with Wikidata and selecting domain-salient relations via Personalized PageRank. To overcome knowledge-base incompleteness, we contribute an automated, character-grounded augmentation pipeline that mines verbatim facts from source articles. Evaluating twenty-one open-weight LLMs against both reference regimes reveals that (i) completeness correction systematically raises precision and lowers recall across all models, especially on multi-valued relations; and (ii) factual recall varies widely across subject domains, demonstrating why single-domain evaluation confounds model capability with subject matter. All code, data, and evaluation scripts are publicly released at https://github.com/Rajjaa/MUSART.

2021

Language models have proven to be very useful when adapted to specific domains. Nonetheless, little research has been done on the adaptation of domain-specific BERT models in the French language. In this paper, we focus on creating a language model adapted to French legal text with the goal of helping law professionals. We conclude that some specific tasks do not benefit from generic language models pre-trained on large amounts of data. We explore the use of smaller architectures in domain-specific sub-languages and their benefits for French legal text. We prove that domain-specific pre-trained models can perform better than their equivalent generalised ones in the legal domain. Finally, we release JuriBERT, a new set of BERT models adapted to the French legal domain.