From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification

Manish Gupta, Chaitanya Giri, Jayasimha Talur


Abstract
Large language models (LLMs) struggle to classify text into taxonomies with many semantically similar labels, as the distinctions are domain-specific and not captured by pre-training. To handle large label spaces, a common approach retrieves top-K candidate labels by embedding similarity and prompt the LLM to choose among them. However, top-K retrieval reduces the number of candidates but does not help the model tell similar ones apart. When two similar labels both appear as candidates, the model lacks the signal to choose correctly between them. We propose a framework that (1) identifies which label pairs the model struggles to distinguish, (2) expands the candidate set to include confusable labels, and (3) generates targeted rules to differentiate between similar candidates. The framework requires no fine-tuning, and the generated rules transfer to smaller, cheaper models. On three benchmarks (WOS, Flipkart, LEDGAR), our approach improves Macro F1 by up to 10.0pp over retrieval baselines, with smaller models (2B–20B) gaining up to 11.5pp via cross-model transfer.
Anthology ID:
2026.emnlp-industry.125
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
2149–2164
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.125/
DOI:
Bibkey:
Cite (ACL):
Manish Gupta, Chaitanya Giri, and Jayasimha Talur. 2026. From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 2149–2164, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification (Gupta et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.125.pdf