LoCoDec: Robust Decoding for LLM-based Sequence-Structured Prediction

Chang Liu, Peiyan Wang, Yubo Feng, Libang Wang, Feiyan Jiang, Yan Song


Abstract
Many Natural Language Processing (NLP) tasks, including information extraction, dialogue understanding, and sentiment analysis, can be formulated as structured prediction over sequences or spans. Although Large Language Model (LLM)-based methods have achieved strong performance on standard benchmarks for such tasks, their robustness under natural sentence-level perturbations and their controllability in producing valid structures remain insufficiently studied. In this paper, we propose Locality-aware Constrained Decoding (LoCoDec), a method for LLM-based structured prediction over sequences and spans that formulates decoding as constrained token paraphrasing. LoCoDec combines neighborhood context, which reduces long-text noise, with structural decoding constraints that enforce source-faithful and well-formed outputs. Experiments on sixteen benchmarks of six tasks show that LoCoDec remains competitive in S-F1 and P-F1, while achieving the best R-F1 on all datasets. Ablation results show that removing neighborhood context reduces R-F1 by up to 16.23 points. Overall, LoCoDec improves the robustness and structural consistency of LLM-based structured prediction.
Anthology ID:
2026.emnlp-main.1457
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
33312–33330
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1457/
DOI:
Bibkey:
Cite (ACL):
Chang Liu, Peiyan Wang, Yubo Feng, Libang Wang, Feiyan Jiang, and Yan Song. 2026. LoCoDec: Robust Decoding for LLM-based Sequence-Structured Prediction. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 33312–33330, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
LoCoDec: Robust Decoding for LLM-based Sequence-Structured Prediction (Liu et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1457.pdf