BLUE: Toward Better Language Use in Efficient Vision-Language-Action Models for Autonomous Driving

George Ling, Lijin Yang, Hao Yang, Zhongzhan Huang


Abstract
We present BLUE, a minimal method for better language use in vision-language-action (VLA) models for autonomous driving (AD). Through extensive analysis, we reveal that language matters on only a small fraction of routes, but on those routes it can greatly improve or degrade performance. Generating language at every frame is therefore inefficient, since most computation is spent on frames that do not benefit from language. We further show that pretrained VLA hidden states potentially already encode whether language will benefit a given frame, even though scene complexity and kinematic features alone struggle to predict this. Based on this finding, BLUE trains a lightweight gate on frozen VLA hidden states to decide per frame whether to activate language generation or predict actions directly, without modifying the backbone or requiring additional human annotation. With just a 0.11M-parameter gate, BLUE sets a new state of the art on both benchmarks, achieving 76.2% success rate on Bench2Drive and 36 driving score on Longest6 v2, while delivering 2.54× inference speedup and 8.9% success rate improvement over the backbone. BLUE provides a practical path toward efficient language-augmented AD, showing that VLA models can retain the benefits of language at a fraction of the cost. Our code, data, logs and checkpoints will be fully released.
Anthology ID:
2026.emnlp-main.1894
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
43156–43185
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1894/
DOI:
Bibkey:
Cite (ACL):
George Ling, Lijin Yang, Hao Yang, and Zhongzhan Huang. 2026. BLUE: Toward Better Language Use in Efficient Vision-Language-Action Models for Autonomous Driving. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 43156–43185, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
BLUE: Toward Better Language Use in Efficient Vision-Language-Action Models for Autonomous Driving (Ling et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1894.pdf