@inproceedings{lu-etal-2026-speaking,
title = "Speaking While Listening: A Survey and Empirical Audit of Full-Duplex Spoken Dialogue Systems",
author = "Lu, Jingyu and
Wang, Yuhan and
Luo, Jianming and
Chen, Yifu and
Liang, Tianle and
Ji, Shengpeng and
Jiang, Ziyue and
Yang, Xiaoda and
Zhang, Yu and
Cheng, Xize and
Wen, Chenyuhao and
Pan, Changhao and
Wang, Haoxiao and
Ye, Chen and
Wu, Jian and
Jiang, Xiaoxi and
Jiang, Guanjun and
Zhao, Zhou",
editor = "Lee, Hung-Yi and
Dev, Sunipa and
Li, Jessy and
Elliott, Desmond",
booktitle = "Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing",
month = oct,
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.emnlp-main.1126/",
pages = "25790--25824",
ISBN = "979-8-89176-459-0",
abstract = "More than a dozen spoken dialogue systems have recently claimed to be ``full-duplex,'' yet the term has been used to describe substantially different capabilities. Existing surveys collapse them onto a single axis (cascaded/end-to-end, or engineered/learned) and miss the distinctions that matter most for builders. We argue that much of this ambiguity is taxonomical: current terminology does not specify where duplex decisions are made, which interaction types are supported, or how a system behaves moment by moment. This paper introduces three complementary frameworks: (i) an L,0-L,3 Architectural Hierarchy that locates where duplex decisions are made; (ii) a T x Ix R Interaction Ontology that specifies the temporal relation, user intent, and required system response for each interaction; and (i) a Decision State Machine(LDLE/LISTEN/SPEAK/WAIT/DUAL) that describes how systems move between states.Across published systems and benchmarks, our audit reveals a realization gap: although many architectures can in principle operate in full-duplex states, their observed behavior remains constrained by the interaction patterns represented in training and evaluation.We identify the open-source-versus-industrial data gap, as well as the still-unrealized goal of L3 representation-level modeling, as key frontiers for future research on full-duplex dialogue. The related material is available at https://github.com/MM-Speech/DuplexSurvey."
}<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lu-etal-2026-speaking">
<titleInfo>
<title>Speaking While Listening: A Survey and Empirical Audit of Full-Duplex Spoken Dialogue Systems</title>
</titleInfo>
<name type="personal">
<namePart type="given">Jingyu</namePart>
<namePart type="family">Lu</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yuhan</namePart>
<namePart type="family">Wang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jianming</namePart>
<namePart type="family">Luo</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yifu</namePart>
<namePart type="family">Chen</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Tianle</namePart>
<namePart type="family">Liang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Shengpeng</namePart>
<namePart type="family">Ji</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Ziyue</namePart>
<namePart type="family">Jiang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Xiaoda</namePart>
<namePart type="family">Yang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yu</namePart>
<namePart type="family">Zhang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Xize</namePart>
<namePart type="family">Cheng</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Chenyuhao</namePart>
<namePart type="family">Wen</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Changhao</namePart>
<namePart type="family">Pan</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Haoxiao</namePart>
<namePart type="family">Wang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Chen</namePart>
<namePart type="family">Ye</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jian</namePart>
<namePart type="family">Wu</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Xiaoxi</namePart>
<namePart type="family">Jiang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Guanjun</namePart>
<namePart type="family">Jiang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Zhou</namePart>
<namePart type="family">Zhao</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<originInfo>
<dateIssued>2026-10</dateIssued>
</originInfo>
<typeOfResource>text</typeOfResource>
<relatedItem type="host">
<titleInfo>
<title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
</titleInfo>
<name type="personal">
<namePart type="given">Hung-Yi</namePart>
<namePart type="family">Lee</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Sunipa</namePart>
<namePart type="family">Dev</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jessy</namePart>
<namePart type="family">Li</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Desmond</namePart>
<namePart type="family">Elliott</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<originInfo>
<publisher>Association for Computational Linguistics</publisher>
<place>
<placeTerm type="text">Budapest, Hungary</placeTerm>
</place>
</originInfo>
<genre authority="marcgt">conference publication</genre>
<identifier type="isbn">979-8-89176-459-0</identifier>
</relatedItem>
<abstract>More than a dozen spoken dialogue systems have recently claimed to be “full-duplex,” yet the term has been used to describe substantially different capabilities. Existing surveys collapse them onto a single axis (cascaded/end-to-end, or engineered/learned) and miss the distinctions that matter most for builders. We argue that much of this ambiguity is taxonomical: current terminology does not specify where duplex decisions are made, which interaction types are supported, or how a system behaves moment by moment. This paper introduces three complementary frameworks: (i) an L,0-L,3 Architectural Hierarchy that locates where duplex decisions are made; (ii) a T x Ix R Interaction Ontology that specifies the temporal relation, user intent, and required system response for each interaction; and (i) a Decision State Machine(LDLE/LISTEN/SPEAK/WAIT/DUAL) that describes how systems move between states.Across published systems and benchmarks, our audit reveals a realization gap: although many architectures can in principle operate in full-duplex states, their observed behavior remains constrained by the interaction patterns represented in training and evaluation.We identify the open-source-versus-industrial data gap, as well as the still-unrealized goal of L3 representation-level modeling, as key frontiers for future research on full-duplex dialogue. The related material is available at https://github.com/MM-Speech/DuplexSurvey.</abstract>
<identifier type="citekey">lu-etal-2026-speaking</identifier>
<location>
<url>https://aclanthology.org/2026.emnlp-main.1126/</url>
</location>
<part>
<date>2026-10</date>
<extent unit="page">
<start>25790</start>
<end>25824</end>
</extent>
</part>
</mods>
</modsCollection>
%0 Conference Proceedings
%T Speaking While Listening: A Survey and Empirical Audit of Full-Duplex Spoken Dialogue Systems
%A Lu, Jingyu
%A Wang, Yuhan
%A Luo, Jianming
%A Chen, Yifu
%A Liang, Tianle
%A Ji, Shengpeng
%A Jiang, Ziyue
%A Yang, Xiaoda
%A Zhang, Yu
%A Cheng, Xize
%A Wen, Chenyuhao
%A Pan, Changhao
%A Wang, Haoxiao
%A Ye, Chen
%A Wu, Jian
%A Jiang, Xiaoxi
%A Jiang, Guanjun
%A Zhao, Zhou
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F lu-etal-2026-speaking
%X More than a dozen spoken dialogue systems have recently claimed to be “full-duplex,” yet the term has been used to describe substantially different capabilities. Existing surveys collapse them onto a single axis (cascaded/end-to-end, or engineered/learned) and miss the distinctions that matter most for builders. We argue that much of this ambiguity is taxonomical: current terminology does not specify where duplex decisions are made, which interaction types are supported, or how a system behaves moment by moment. This paper introduces three complementary frameworks: (i) an L,0-L,3 Architectural Hierarchy that locates where duplex decisions are made; (ii) a T x Ix R Interaction Ontology that specifies the temporal relation, user intent, and required system response for each interaction; and (i) a Decision State Machine(LDLE/LISTEN/SPEAK/WAIT/DUAL) that describes how systems move between states.Across published systems and benchmarks, our audit reveals a realization gap: although many architectures can in principle operate in full-duplex states, their observed behavior remains constrained by the interaction patterns represented in training and evaluation.We identify the open-source-versus-industrial data gap, as well as the still-unrealized goal of L3 representation-level modeling, as key frontiers for future research on full-duplex dialogue. The related material is available at https://github.com/MM-Speech/DuplexSurvey.
%U https://aclanthology.org/2026.emnlp-main.1126/
%P 25790-25824
Markdown (Informal)
[Speaking While Listening: A Survey and Empirical Audit of Full-Duplex Spoken Dialogue Systems](https://aclanthology.org/2026.emnlp-main.1126/) (Lu et al., EMNLP 2026)
ACL
- Jingyu Lu, Yuhan Wang, Jianming Luo, Yifu Chen, Tianle Liang, Shengpeng Ji, Ziyue Jiang, Xiaoda Yang, Yu Zhang, Xize Cheng, Chenyuhao Wen, Changhao Pan, Haoxiao Wang, Chen Ye, Jian Wu, Xiaoxi Jiang, Guanjun Jiang, and Zhou Zhao. 2026. Speaking While Listening: A Survey and Empirical Audit of Full-Duplex Spoken Dialogue Systems. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 25790–25824, Budapest, Hungary. Association for Computational Linguistics.