Beyond Dense Retrieval: A Structured-Semantic Approach for Conversation Analytics

Abinesh Kanagarajan, Chetan Aggarwal, Anjali Jain


Abstract
Large Language Model (LLM) based conversational systems generate vast interaction logs containing critical diagnostic signals such as failure modes and safety violations. However, the cross-turn dependencies, implicit state, and topic drift inherent to these conversations defeat dense retrieval, whose lone embedding cannot concurrently encode discrete structured constraints (e.g., refusal type) and open-ended semantic content. Our central claim is conceptual: conversational retrieval must span two complementary information planes, structured metadata and semantic content, and these are best reconciled by a unified hybrid representation that jointly supports filtering over structured metadata fields and vector similarity search over unbounded open-ended fields. We instantiate this as the Extract-Annotate-Query (EAQ) framework, which derives a metadata taxonomy, applies LLM-driven annotation, and compiles natural-language queries into hybrid graph operations. On the CDR-Benchmark, EAQ lifts NDCG@10 by 30.8% over the strongest dense baseline, with consistent gains across five analytical categories and two external datasets.
Anthology ID:
2026.emnlp-industry.207
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
3466–3489
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.207/
DOI:
Bibkey:
Cite (ACL):
Abinesh Kanagarajan, Chetan Aggarwal, and Anjali Jain. 2026. Beyond Dense Retrieval: A Structured-Semantic Approach for Conversation Analytics. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 3466–3489, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Beyond Dense Retrieval: A Structured-Semantic Approach for Conversation Analytics (Kanagarajan et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.207.pdf