From Evidence to Exact Answers: Copy-Preserving Compression and Conversion-Aware Calibration for Frozen SLM RAG

Dong Ho Kang, HyeonJeong Cha, Daein Weon


Abstract
Frozen small language model (SLM) readers in retrieval-augmented generation (RAG) can fail even after retrieval has found answer-bearing evidence: on five LongBench tasks with LFM2.5-Instruct 1.2B, Raw RAG with its default prompt converts only 43% of answer-bearing predictions into exact matches. We treat frozen SLM RAG as a two-stage problem—copy-preserving evidence compression and conversion-aware deployment calibration—and instantiate each stage with one system. PEC-HOP is a non-parametric pointer-preserved evidence compressor; the Minimal Copy-preserving Evidence Compass (MCE-Compass) is a dev-time calibrator that selects an (interface, prompt) pair per (reader, task) cell, calibrated offline from ~50 labeled examples per cell for fixed deployments. In a matched-prompt head-to-head on three multi-hop LongBench tasks × three frozen readers, PEC-HOP is Pareto-best among compressors, with +4.28 pp macro EM over LLMLingua-2 and +1.61 pp over Provence at 2.8–4.8× faster median latency and no trained compressor on the critical path. Across five tasks, MCE-Compass improves macro EM over Raw RAG by +9.36, +15.39, and +2.39 pp on LFM2.5-Instruct 1.2B, Qwen3-4B, and Gemma-4-E4B at 2.02×, 5.69×, and 1.34× latency speedups, and beats the single globally dev-selected policy. A 20-partition dev-size sweep and a 66,581-article HotpotQA retrieval stress test indicate the gains are not single-split or bundled-context artifacts.
Anthology ID:
2026.emnlp-main.849
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
19678–19694
Language:
URL:
https://aclanthology.org/2026.emnlp-main.849/
DOI:
Bibkey:
Cite (ACL):
Dong Ho Kang, HyeonJeong Cha, and Daein Weon. 2026. From Evidence to Exact Answers: Copy-Preserving Compression and Conversion-Aware Calibration for Frozen SLM RAG. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 19678–19694, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
From Evidence to Exact Answers: Copy-Preserving Compression and Conversion-Aware Calibration for Frozen SLM RAG (Kang et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.849.pdf