How Do Document Parsers Break? Auditing Structural Vulnerability in Document Intelligence

Yue Chen, Yihao Wang, Ziyi Tang, Yongsen Zheng, Keze Wang


Abstract
Document Layout Analysis (DLA) pipelines provide structured page representations for retrieval-augmented generation, long-document question answering, and related applications. Yet their robustness evaluation remains largely area-centric. We identify this *Footprint Bias* and propose **ProSA**, a lightweight output-level auditing framework that decouples controlled probing, policy-driven targeting, and structure-aware diagnosis. ProSA combines Block-level Structural Loss Rate (**B-SLR**), granularity-aware exposure descriptors, and pathway attribution to analyze where structural identity is lost, at what exposure granularity failures emerge, and how failures propagate. Across MinerU and PP-StructureV3 on 1,000 pages, affected area weakly tracks perturbation-induced OCR instability (R2 = 0.384/0.110), whereas B-SLR aligns much more closely with it (R2 = 0.727/0.916). Exposure descriptors further separate occlusion- and topology-dominant pathways, while matched-footprint structural probes cause much larger downstream QA/retrieval drops than area-matched erasure. These results shift DLA robustness evaluation from footprint-based measurement toward structure-aware vulnerability auditing.
Anthology ID:
2026.emnlp-main.1921
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
43751–43773
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1921/
DOI:
Bibkey:
Cite (ACL):
Yue Chen, Yihao Wang, Ziyi Tang, Yongsen Zheng, and Keze Wang. 2026. How Do Document Parsers Break? Auditing Structural Vulnerability in Document Intelligence. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 43751–43773, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
How Do Document Parsers Break? Auditing Structural Vulnerability in Document Intelligence (Chen et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1921.pdf