@inproceedings{zhao-etal-2026-monolithic,
title = "From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale",
author = "Zhao, Cen and
Wang, Peng and
Shi, Chuan and
Zhang, Yufeng and
Lyu, Ying and
Ren, Wanmeng and
Xue, Robert and
Cheng, Claire and
Mehdad, Yashar",
editor = "Atanasova, Pepa and
Haffari, Gholamreza and
Wang, Minghan",
booktitle = "Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track",
month = oct,
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.emnlp-industry.153/",
pages = "2591--2607",
ISBN = "979-8-89176-460-6",
abstract = "Conversational assistants can blend retrieval, action selection, escalation, and wording in one model path, or separate those roles. We report a production migration of a conversational assistant for customer support at a large accommodation marketplace (millions of conversations per month, 11 languages, 10-second P90). Dynamic Response (DR) replaces a single Qwen3-235B-A22B blended responder with a bounded ReAct orchestrator over typed tools plus a smaller generator that writes from a backend-validated context contract. Because the migration also changed prompts, alignment, and serving, we attribute each effect to its cause and claim as architecture effects only those measured on identical replayed turns: typed entity selection moves the reservation selector to a precision-first operating point (precision 8.3{\%} to 89.1{\%} at recall 75.2{\%} to 67.3{\%}), and typed action IDs with a membership check remove observed structured-action hallucination (2.14{\%} to 0.0{\%}). A low-ramp A/B test reproduces the replay escalation reductions: hard-escalation responses fall 5.60{\%} to 3.08{\%} and soft-escalation responses 9.56{\%} to 2.49{\%} while production handoff volume holds roughly steady, and self-solve is directional (+5.1 points, 95{\%} CI [-2, +12]). Serving optimizations cut orchestrator P90 from 3.87s to 2.24s on a GPU footprint reduced by roughly one-third, and self-hosting reduces estimated annual model-serving cost by more than an order of magnitude."
}<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zhao-etal-2026-monolithic">
<titleInfo>
<title>From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale</title>
</titleInfo>
<name type="personal">
<namePart type="given">Cen</namePart>
<namePart type="family">Zhao</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Peng</namePart>
<namePart type="family">Wang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Chuan</namePart>
<namePart type="family">Shi</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yufeng</namePart>
<namePart type="family">Zhang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Ying</namePart>
<namePart type="family">Lyu</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Wanmeng</namePart>
<namePart type="family">Ren</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Robert</namePart>
<namePart type="family">Xue</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Claire</namePart>
<namePart type="family">Cheng</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yashar</namePart>
<namePart type="family">Mehdad</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<originInfo>
<dateIssued>2026-10</dateIssued>
</originInfo>
<typeOfResource>text</typeOfResource>
<relatedItem type="host">
<titleInfo>
<title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track</title>
</titleInfo>
<name type="personal">
<namePart type="given">Pepa</namePart>
<namePart type="family">Atanasova</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Gholamreza</namePart>
<namePart type="family">Haffari</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Minghan</namePart>
<namePart type="family">Wang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<originInfo>
<publisher>Association for Computational Linguistics</publisher>
<place>
<placeTerm type="text">Budapest, Hungary</placeTerm>
</place>
</originInfo>
<genre authority="marcgt">conference publication</genre>
<identifier type="isbn">979-8-89176-460-6</identifier>
</relatedItem>
<abstract>Conversational assistants can blend retrieval, action selection, escalation, and wording in one model path, or separate those roles. We report a production migration of a conversational assistant for customer support at a large accommodation marketplace (millions of conversations per month, 11 languages, 10-second P90). Dynamic Response (DR) replaces a single Qwen3-235B-A22B blended responder with a bounded ReAct orchestrator over typed tools plus a smaller generator that writes from a backend-validated context contract. Because the migration also changed prompts, alignment, and serving, we attribute each effect to its cause and claim as architecture effects only those measured on identical replayed turns: typed entity selection moves the reservation selector to a precision-first operating point (precision 8.3% to 89.1% at recall 75.2% to 67.3%), and typed action IDs with a membership check remove observed structured-action hallucination (2.14% to 0.0%). A low-ramp A/B test reproduces the replay escalation reductions: hard-escalation responses fall 5.60% to 3.08% and soft-escalation responses 9.56% to 2.49% while production handoff volume holds roughly steady, and self-solve is directional (+5.1 points, 95% CI [-2, +12]). Serving optimizations cut orchestrator P90 from 3.87s to 2.24s on a GPU footprint reduced by roughly one-third, and self-hosting reduces estimated annual model-serving cost by more than an order of magnitude.</abstract>
<identifier type="citekey">zhao-etal-2026-monolithic</identifier>
<location>
<url>https://aclanthology.org/2026.emnlp-industry.153/</url>
</location>
<part>
<date>2026-10</date>
<extent unit="page">
<start>2591</start>
<end>2607</end>
</extent>
</part>
</mods>
</modsCollection>
%0 Conference Proceedings
%T From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale
%A Zhao, Cen
%A Wang, Peng
%A Shi, Chuan
%A Zhang, Yufeng
%A Lyu, Ying
%A Ren, Wanmeng
%A Xue, Robert
%A Cheng, Claire
%A Mehdad, Yashar
%Y Atanasova, Pepa
%Y Haffari, Gholamreza
%Y Wang, Minghan
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-460-6
%F zhao-etal-2026-monolithic
%X Conversational assistants can blend retrieval, action selection, escalation, and wording in one model path, or separate those roles. We report a production migration of a conversational assistant for customer support at a large accommodation marketplace (millions of conversations per month, 11 languages, 10-second P90). Dynamic Response (DR) replaces a single Qwen3-235B-A22B blended responder with a bounded ReAct orchestrator over typed tools plus a smaller generator that writes from a backend-validated context contract. Because the migration also changed prompts, alignment, and serving, we attribute each effect to its cause and claim as architecture effects only those measured on identical replayed turns: typed entity selection moves the reservation selector to a precision-first operating point (precision 8.3% to 89.1% at recall 75.2% to 67.3%), and typed action IDs with a membership check remove observed structured-action hallucination (2.14% to 0.0%). A low-ramp A/B test reproduces the replay escalation reductions: hard-escalation responses fall 5.60% to 3.08% and soft-escalation responses 9.56% to 2.49% while production handoff volume holds roughly steady, and self-solve is directional (+5.1 points, 95% CI [-2, +12]). Serving optimizations cut orchestrator P90 from 3.87s to 2.24s on a GPU footprint reduced by roughly one-third, and self-hosting reduces estimated annual model-serving cost by more than an order of magnitude.
%U https://aclanthology.org/2026.emnlp-industry.153/
%P 2591-2607
Markdown (Informal)
[From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale](https://aclanthology.org/2026.emnlp-industry.153/) (Zhao et al., EMNLP 2026)
ACL
- Cen Zhao, Peng Wang, Chuan Shi, Yufeng Zhang, Ying Lyu, Wanmeng Ren, Robert Xue, Claire Cheng, and Yashar Mehdad. 2026. From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 2591–2607, Budapest, Hungary. Association for Computational Linguistics.