RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents

Mingxuan Zhang, Xiaowen Wang, Anupma Sharan, Zhengyi Chen, Chenyu Diana Zhang, Shanshan Yang, Chittibabu Pacharu


Abstract
Effective troubleshooting agents in enterprise customer support depend on retrieving actionable guidance from similar historical resolved cases, yet existing retrieval-augmented generation (RAG) systems treat support cases as static documents and overlook their multi-stage, stateful nature. We introduce RAFT (Retrieval-Augmented Framework for Troubleshooting Agents), a stateful RAG framework that abstracts each resolved case into a directed chain of timeline entries and retrieves at the entry level, surfacing cases whose intermediate states match the active case and returning the parent-case trajectory anchored at the matched state; an optional case-level graph links cases by shared root cause and resolution. We evaluate this retrieval layer directly, which, unlike evaluating a full agent system, requires no production deployment. Because public multi-stage troubleshooting data is extremely rare, we pair a synthetic benchmark built from Microsoft Learn Windows Server documentation with real Apache Jira issues carrying human-created duplicate labels. RAFT improves Case Hit over vanilla RAG and GraphRAG baselines at every stage of case progress, with statistically significant gains over the strongest baseline; the Jira results provide directional evidence that the advantage transfers to real case histories. We release our benchmark, implementation, and the Apache Jira evaluation set.
Anthology ID:
2026.emnlp-industry.21
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
315–328
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.21/
DOI:
Bibkey:
Cite (ACL):
Mingxuan Zhang, Xiaowen Wang, Anupma Sharan, Zhengyi Chen, Chenyu Diana Zhang, Shanshan Yang, and Chittibabu Pacharu. 2026. RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 315–328, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents (Zhang et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.21.pdf