Business Compromise Detection with Agentic AI and LLM-driven Knowledge Discovery

Diego Palma, Kyu Bin Kim, Zhen Han, Allbright Dsouza, Zhiyuan Liu


Abstract
The detection of compromised business ad accounts is a critical challenge in digital advertising, as attackers exploit hijacked accounts to launch fraudulent campaigns. Large Language Model (LLM) agents show promise for integrity violations, but we find that they cause business friction through hallucinated mistakes on hard cases. Through a deployment study we observe that an autonomous LLM agent is a strong, recall-heavy signal extractor but an unreliable final arbiter for ambiguous compromise decisions, conceding substantial precision. We therefore pair the agent with a knowledge-based decision stage: the agent investigates an account and emits a structured, interpretable signal vector, and a neuro-symbolic stage makes the decision in three layers, symbolic rules discovered automatically with an Inductive Logic Programming (FOIL-IE) algorithm, a Naïve Bayes layer that calibrates those rules, and a data-tuned contradiction layer. We evaluate on a large oversampled general population (compromise oversampled, used because hard-case labels are scarce) and a realistic, low-prevalence production sample with subject-matter-expert (SME) labels, benchmarking against tree-ensemble baselines under identical conditions. The knowledge stage restores much of the precision lost by the agent (up to doubling it). On realistic low-prevalence data the full three-layer stage exceeds human expert reviewers and tree-ensemble baselines, with each layer contributing; its rules encode domain priors and generalize from few labels where ensembles overfit, while remaining interpretable and auditable.
Anthology ID:
2026.emnlp-industry.119
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
2045–2057
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.119/
DOI:
Bibkey:
Cite (ACL):
Diego Palma, Kyu Bin Kim, Zhen Han, Allbright Dsouza, and Zhiyuan Liu. 2026. Business Compromise Detection with Agentic AI and LLM-driven Knowledge Discovery. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 2045–2057, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Business Compromise Detection with Agentic AI and LLM-driven Knowledge Discovery (Palma et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.119.pdf