Lisa Reichenbach

Author directory

2026

Practitioners across industries increasingly use large language models to extract information from unstructured documents such as financial filings, legal contracts, and sustainability reports, yet typically rely on generic online recommendations for prompt design that lack empirical grounding in their target setting. We address this gap with a practical methodology for systematically varying individual prompt components (prompt structure, contextual information, and model selection) and measuring their effect on extraction quality. We use greenhouse gas emissions extraction from corporate sustainability reports as a case study of fixed numeric extraction from high-stakes PDF documents that vary widely in structure and presentation and are dispersed across company websites. Instantiated in a retrieval-augmented generation pipeline, our evaluation shows that best practices from industry-facing prompting guides do not always hold: a structured prompt incorporating roles, definitions, extraction rules, and JSON formatting unexpectedly underperforms a simpler question-style prompt, while the benefit of adding tabular context proves both prompt- and model-specific. We condense our evaluation procedure and findings into a practitioner’s decision guide for systematically testing design choices on task-specific data rather than adopting generic recommendations by default.