Yash Raj Singh
2026
CrossOracle: An Agentic Framework for Real-Time Expert Personas with Parallelized Acoustic Synthesis
Yash Raj Singh
Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Yash Raj Singh
Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue
We present CrossOracle, a production-deployed spoken dialogue system for real-time Hinglish (Hindi-English code-switched) voice interaction across eleven Expert Persona identities. The system addresses the latency bottleneck of cascaded ASR-LLM-TTS pipelines through three contributions: (1) a parallelized First-In-First-Stream TTS worker that overlaps synthesis with ongoing LLM generation, mathematically ensuring gapless audio playback; (2) a language-aware semantic boundary heuristic that flushes text chunks at Hindi danda and Latin punctuation boundaries, preserving prosodic integrity across code-switched utterances; and (3) a deterministic AbortController-based barge-in protocol that resolves user interruptions in under 1 ms. Empirical evaluation of live conversational logs demonstrates a mean TTS Real-Time Factor of 0.107 and a 100% barge-in resolution rate. The live system is deployed at crossoracle.io and supports persona switching, custom persona creation from natural language descriptions, and robust Hinglish ASR error tolerance via a Devanagari ratio mishear filter. The demonstration invites attendees to hold unscripted Hinglish conversations, stress-test the barge-in protocol, and construct novel personas on-the-fly.