Hallucination is a persistent issue affecting all large language Models (LLMs), particularly within low-resource languages such as Persian. PerHalluEval (Persian Hallucination Evaluation) is the first dynamic hallucination evaluation benchmark tailored for the Persian language. Our benchmark leverages a three-stage LLM-driven pipeline, augmented with human validation, to generate plausible answers and summaries regarding QA and summarization tasks, focusing on detecting extrinsic and intrinsic hallucinations. Moreover, we used the log probabilities of generated tokens to select the most believable hallucinated instances. In addition, we engaged human annotators to highlight Persian-specific contexts in the QA dataset in order to evaluate LLMs’ performance on content specifically related to Persian culture. Our evaluation of 12 LLMs, including open- and closed-source models using PerHalluEval, revealed that the models generally struggle in detecting hallucinated Persian text. We showed that providing external knowledge, i.e., the original document for the summarization task, could mitigate hallucination partially. Furthermore, there was no significant difference in terms of hallucination when comparing LLMs specifically trained for Persian with others.

Knowledge Graph Question Answering (KGQA) is particularly challenging for Persian, a complex and low-resource language with significant ambiguity. To address this, we introduce PeMA-KGQA (Persian Multi-Agent Knowledge Graph Question Answering), the first multi-agent LLM framework for Persian KGQA, arguing that an agent-based approach is essential for robust reasoning. Our system operates on FarsBase and is evaluated on both the ParsSimpleQA (one-hop) and PeCoQ (compositional) benchmarks. The framework employs a plan-and-execute paradigm where a dedicated Planner agent first decomposes a question into a structured executable plan using a toolbox of operators. Subsequent specialized agents—an Entity Grounder, a Relation Selector, and a Path Scorer—then execute this plan, effectively grounding mentions, selecting relations, and scoring reasoning paths to produce the final answer. This multi-agent decomposition provides a transparent and interpretable reasoning trace. PeMA-KGQA sets a new state-of-the-art on ParsSimpleQA with 21.14% improvement in Hits@5 and achieves 80.67% in Hits@5 on the complex multi-hop questions of PeCoQ.