Retrieval-Conditional Parsing Score (RCPS): Choosing Document Parsers by Retrieval, Not by Appearance

Sang-Woo Son, Hyeong-seob Kim, Hyeonsang Kim, Hyun-woo Cho, Jinmo Kim


Abstract
Document RAG retrieves from parser outputs, yet parsers are commonly selected by intrinsic measures of transcription fidelity and boundary quality rather than by retrieval performance. We introduce the Retrieval-Conditional Parsing Score (RCPS), a training-free protocol that ranks parser–chunker combinations on a fixed held-out Q–A retrieval probe. RCPS averages mean reciprocal rank (MRR) across specified retrievers and retrieval depths, using the same format-normalised exact-span relevance rule for every candidate. On KoGovDoc-RAG, table-enabled MinerU-on has a higher Boundary Clarity score than our production parser, Prod (0.713 versus 0.610). Yet its Hit@1 is 42.6 percentage points lower (0.123 versus 0.549). Across five parsers evaluated on all 294 pages, RCPS ranges from 0.137 to 0.584. With Prod fixed, the four chunkers score between 0.535 and 0.593. A retriever-free coverage diagnostic finds no normalised exact match for 20.2% of reference spans in Prod output. No tested chunker splits more than 2.3%. On source-aligned OHR-Bench data, semantic noise lowers retrieval while Boundary Clarity remains stable or changes non-monotonically. In a secondary parser-training study, two direct preference optimization checkpoints improve Hit@5 by only 0.95 and 1.15 percentage points over Prod on an audited compatibility subset; a matched edit-distance control performs similarly. The workflow therefore selects with RCPS, diagnoses with coverage, and trains only if needed. We release the 663-Q–A KoGovDoc-RAG probe and an RCPS reference implementation.
Anthology ID:
2026.emnlp-industry.90
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1531–1544
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.90/
DOI:
Bibkey:
Cite (ACL):
Sang-Woo Son, Hyeong-seob Kim, Hyeonsang Kim, Hyun-woo Cho, and Jinmo Kim. 2026. Retrieval-Conditional Parsing Score (RCPS): Choosing Document Parsers by Retrieval, Not by Appearance. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1531–1544, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Retrieval-Conditional Parsing Score (RCPS): Choosing Document Parsers by Retrieval, Not by Appearance (Son et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.90.pdf