2026

With the emergence of large language models (LLMs), there is an expectation that they can effectively extract structured information from complex, real-world documents. However, most LLMs generate paragraph-style answers that are chaotic, disorganized, and untraceable. To bridge this gap, we introduce the Arranged and Organized Extraction Benchmark (AOE), a new bilingual benchmark featuring long-form documents (171k tokens on average) designed to systematically evaluate the ability of LLMs to comprehend fragmented information and reconstruct it into an organized table. Unlike conventional text-to-table tasks, which rely on fixed schema and narrow domains, AOE includes 12 carefully designed tasks across four diverse domains, requiring models to generate context-specific schema tailored to varied input queries. In our experiments, we evaluated 12 open-source and proprietary LLMs. Even the most advanced models struggle significantly.