Irvin Cardoza
Author directory2026
Topic Matching in the Wild: Benchmark and Lessons from Real-World ASR Transcripts
Saman Rahbar | Xiliang Zhu | Irvin Cardoza | David Rossouw
Proceedings of the 11th Workshop on Natural User-generated Text
Saman Rahbar | Xiliang Zhu | Irvin Cardoza | David Rossouw
Proceedings of the 11th Workshop on Natural User-generated Text
In contact centers, real-time agent-assist tools determine, for each of many predefined topics, whether a live customer utterance is relevant and display a coaching card to the agent when it is. The input is noisy and challenging: ASR (Automatic Speech Recognition) transcripts of spontaneous phone conversations, which can be unclear, repetitive, and mostly lack punctuation. To systematically study this real-world task, we curate a human-annotated topic-utterance judgments dataset sourced from real call-center transcripts. We compare three types of matchers: a regex-based baseline, zero-shot sentence-embedding encoders, and Gemini-based LLM matchers. In addition, two types of topic representations are studied in our benchmark: keyphrases and natural language description. Our empirical experiments highlight the superior performance of lightweight LLM matchers when equipped with natural language descriptions.