Closing the Bilingual Gap: Cross-lingual Semantic Scaffolding for Fine-Grained Machine Translation Evaluation in Taiwanese Hokkien

You-Wei Wu, En-Shiun Annie Lee, Richard Tzong-Han Tsai


Abstract
Despite the growing adoption of large language models (LLMs) for machine translation (MT) evaluation, reliably detecting fine-grained error spans in low-resource languages remains challenging. A key limitation is that LLMs often lack sufficient cross-lingual knowledge, making it difficult for them to accurately localize errors at the span level. In this work, we introduce Cross-lingual Semantic Scaffolding, a two-part prompting framework that incorporates sentence-level parallel examples and word-level lexical mappings at inference time. We evaluate this approach on five LLMs using controlled ablation experiments on Taiwanese Hokkien. The proposed framework consistently improves performance across all tested models, increasing average Span F1 by +0.175 (+33.6%) and average Span Recall by +0.351 (+75.9%), and shows that the gains are mainly driven by improved error coverage. It also supports AI-assisted human annotation, reducing expert annotation time from 450 seconds to 90.92 seconds per sentence, with 98.2% of AI-proposed spans accepted without modification. These results demonstrate that Cross-lingual Semantic Scaffolding is both an effective detection method and a practical tool for reducing expert annotation costs in low-resource MT evaluation.
Anthology ID:
2026.aacl-short.43
Volume:
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venues:
AACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
664–678
Language:
URL:
https://aclanthology.org/2026.aacl-short.43/
DOI:
Bibkey:
Cite (ACL):
You-Wei Wu, En-Shiun Annie Lee, and Richard Tzong-Han Tsai. 2026. Closing the Bilingual Gap: Cross-lingual Semantic Scaffolding for Fine-Grained Machine Translation Evaluation in Taiwanese Hokkien. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 664–678, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
Closing the Bilingual Gap: Cross-lingual Semantic Scaffolding for Fine-Grained Machine Translation Evaluation in Taiwanese Hokkien (Wu et al., AACL-IJCNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aacl-short.43.pdf