Boris Kundu
Author directory2026
Failure Modes of Imperfect Relational Context in LLM Document Extraction
Boris Kundu | Pooja Mehta
Proceedings of the 3rd Workshop on KNOwledge GRaphs Agentic Systems Interplay (NORA 2026)
Boris Kundu | Pooja Mehta
Proceedings of the 3rd Workshop on KNOwledge GRaphs Agentic Systems Interplay (NORA 2026)
LLM-based extraction pipelines increasingly consume relational context produced upstream, for example by a heuristic linker, a retrieval step, or another model. We study how imperfect relations between document entities affect LLM extraction on FUNSD, a benchmark of scanned forms. Using identical instructions in every condition, four LLMs receive a form’s entities with no relations (Flat), relations from a heuristic linker (Heuristic), gold relations (Oracle), or relations that are almost all wrong (Deranged). In a secondary contrast, gold relations raise accuracy for every model (4.0 to 7.9 percentage points, pp). Heuristic and deranged relations lower accuracy by 8 to 25 pp for three models, while GPT-5 shows significantly smaller effects. Comparing the same questions across conditions shows what lies behind the drops. Wrong edges are associated with answers copied from a different field, and missing edges with more blank answers even though the answer text is still present. These patterns appear under two prompt variants, but the prompts were developed on this corpus and every result is a single generation, so we treat the prompt comparison as exploratory.