Jinyoung Park
Author directoryOther people with similar names: Jinyoung Park
2026
StreamAlign: Streaming Text-Aligned Speech Tokenization
Kang-wook Kim | Jinyoung Park | Jinsoo Kim | Sehun Lee | Tony Woo | Gunhee Kim
Findings of the Association for Computational Linguistics: EMNLP 2026
Kang-wook Kim | Jinyoung Park | Jinsoo Kim | Sehun Lee | Tony Woo | Gunhee Kim
Findings of the Association for Computational Linguistics: EMNLP 2026
Text-aligned speech tokenization methods have emerged to better align speech tokens with LLM token spaces, enabling more effective utilization of pretrained LLMs. However, they rely on offline automatic speech recognition (ASR), leading to two key limitations: (i) the need for complete utterances before tokenization, precluding real-time streaming, and (ii) vocabulary mismatch between ASR and LLMs, which reduces acoustic granularity from the subword to the word level. We introduce StreamAlign, a text-aligned speech tokenization framework that enables streaming tokenization for real-time speech–text joint modeling. StreamAlign performs online speech–text alignment by combining character-level RNN-Transducer alignment with word-level ASR guidance, mitigating ASR–LLM vocabulary mismatch while preserving recognition accuracy. A proactive word boundary classifier anticipates word completion at chunk boundaries, reducing tokenization latency from 560ms to 270 ms. On LibriSpeech, StreamAlign achieves the lowest WER and highest UTMOS among evaluated tokenizers. Furthermore, StreamAlign-SLM, a spoken language model trained on StreamAlign units, demonstrates the strongest overall performance in terms of both acoustic and semantic consistency on the SALMon and spoken StoryCloze benchmarks.