Shiwei ye
Author directory2026
Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction
Tianyun Zhong | Guozhao Mo | Yanjiang Liu | Yihan Chen | Lingdi Kong | Xuanang Chen | Yaojie Lu | Hongyu Lin | Shiwei ye | Shanshan Jiang | Bin Dong | Le Sun | Ben He | Xianpei Han
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Tianyun Zhong | Guozhao Mo | Yanjiang Liu | Yihan Chen | Lingdi Kong | Xuanang Chen | Yaojie Lu | Hongyu Lin | Shiwei ye | Shanshan Jiang | Bin Dong | Le Sun | Ben He | Xianpei Han
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
With the emergence of large language models (LLMs), there is an expectation that they can effectively extract structured information from complex, real-world documents. However, most LLMs generate paragraph-style answers that are chaotic, disorganized, and untraceable. To bridge this gap, we introduce the Arranged and Organized Extraction Benchmark (AOE), a new bilingual benchmark featuring long-form documents (171k tokens on average) designed to systematically evaluate the ability of LLMs to comprehend fragmented information and reconstruct it into an organized table. Unlike conventional text-to-table tasks, which rely on fixed schema and narrow domains, AOE includes 12 carefully designed tasks across four diverse domains, requiring models to generate context-specific schema tailored to varied input queries. In our experiments, we evaluated 12 open-source and proprietary LLMs. Even the most advanced models struggle significantly.