HALT: Hallucination Assessment via Log-probs as Time series

Ahmad Shapiro, Karan Taneja, Ashok Goel


Abstract
Detecting hallucinations in Large Language Model (LLM) outputs is critical for safe deployment, yet existing methods require white-box access to internals, depend on multiple generations or auxiliary judge models, or collapse token-level uncertainty into scalar summaries that discard temporal structure. We introduce HALT, a lightweight grey-box detector that treats the top-20 log-probabilities of an LLM generation as a multivariate time series and processes them with a small bidirectional GRU (5M parameters), in a single forward pass and using no model internals. We evaluate on HUB, a unified protocol reorganizing existing hallucination datasets into ten capability clusters spanning factual and logical failure modes. HALT matches or surpasses Lettuce, a fine-tuned ModernBERT-base detector, on macro-F1 while using ∼30× fewer parameters; a controlled comparison against a text classifier trained on HALT’s own data confirms this advantage comes from the log-probability signal itself rather than from differences in training data, and the same recipe applies across eight LLM families from 360M to 70B parameters.
Anthology ID:
2026.emnlp-main.399
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
9174–9190
Language:
URL:
https://aclanthology.org/2026.emnlp-main.399/
DOI:
Bibkey:
Cite (ACL):
Ahmad Shapiro, Karan Taneja, and Ashok Goel. 2026. HALT: Hallucination Assessment via Log-probs as Time series. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 9174–9190, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
HALT: Hallucination Assessment via Log-probs as Time series (Shapiro et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.399.pdf