Oddballness: universal anomaly detection with language models

Filip Gralinski, Ryszard Staruch, Krzysztof Jurkiewicz


Abstract
We present a new method to detect anomalies in texts (in general: in sequences of any data), using language models, in a totally unsupervised manner. The method considers probabilities (likelihoods) generated by a language model, but instead of focusing on low-likelihood tokens, it considers a new metric defined in this paper: oddballness. Oddballness measures how “strange” a given token is according to the language model. We demonstrate in grammatical error detection tasks (a specific case of text anomaly detection) that oddballness is better than just considering low-likelihood events, if a totally unsupervised setup is assumed.
Anthology ID:
2025.coling-main.183
Volume:
Proceedings of the 31st International Conference on Computational Linguistics
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2683–2689
Language:
URL:
https://aclanthology.org/2025.coling-main.183/
DOI:
Bibkey:
Cite (ACL):
Filip Gralinski, Ryszard Staruch, and Krzysztof Jurkiewicz. 2025. Oddballness: universal anomaly detection with language models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 2683–2689, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
Oddballness: universal anomaly detection with language models (Gralinski et al., COLING 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.coling-main.183.pdf