Silence over Error: Risk-Averse Navigation Landmark Extraction via Progressive MLLM Alignment

Xin Yang, Ziqi Cao, Changning Shao, Junhao Lin, Fanxiang Zeng, Yue Liu


Abstract
Existing pedestrian navigation systems often provide direction-only instructions with limited visual confirmation at complex intersections or along long straight paths. We study coordinate-free landmark extraction from dashcam image sequences. The model returns a set of landmarks with structured attributes (e.g., category and description), or abstains when visual evidence is insufficient. Although general MLLMs can recognize street-scene objects, they do not reliably follow expert rules for filtering ambiguous candidates. Moreover, a false landmark can mislead a pedestrian, making precision more important than exhaustive coverage in this task. We therefore develop a progressive alignment framework, supported by a landmark annotation and augmentation pipeline that converts navigation experts’ selection criteria into staged supervision signals. We further propose a hard-gated bipartite matcher for box-free evaluation and a navigation-aware risk-sensitive reward that penalizes false positives more heavily than abstentions without inducing blanket silence. On the held-out benchmark, our model achieves 92.29% precision while maintaining practical coverage, outperforming strong MLLM baselines. A 30-day A/B test with 2M users on the Amap platform further shows that landmark-augmented instructions reduce departure-confirmation latency by 23.5%. Together, these results demonstrate that risk-aware alignment enables reliable, abstention-capable landmark extraction for navigation.
Anthology ID:
2026.findings-emnlp.1478
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
31455–31476
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1478/
DOI:
Bibkey:
Cite (ACL):
Xin Yang, Ziqi Cao, Changning Shao, Junhao Lin, Fanxiang Zeng, and Yue Liu. 2026. Silence over Error: Risk-Averse Navigation Landmark Extraction via Progressive MLLM Alignment. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 31455–31476, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Silence over Error: Risk-Averse Navigation Landmark Extraction via Progressive MLLM Alignment (Yang et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1478.pdf