Improving Information Extraction with Learned Queries

Omar Sharif, Soroush Vosoughi, Nikhil Singh


Abstract
When information extraction fails, a natural instinct is to improve the model doing it: for example, by scaling it up or refining its reasoning. In this paper, we show that another part of the pipeline matters at least as much: the queries used to elicit this information. Across four clinical benchmarks and five LLMs, improving the question design alone raises performance by 18.6 F1-score points, i.e. more than using larger extraction models. To make such question design learnable, we introduce List of Questions (LoQ), which generates document-specific question sets, and FeedQ, a feedback-driven optimization method that iteratively refines questions against extraction outcomes. The resulting optimized questions can be used to train lightweight generators: with fine-tuning, 4B-parameter models match or outperform expert-derived baselines and substantially exceed the performance of much larger untuned models. We release a dataset of 12,820 optimized questions to support a broader shift in information extraction research toward treating question design as a first-class problem.
Anthology ID:
2026.findings-emnlp.800
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
16952–16972
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.800/
DOI:
Bibkey:
Cite (ACL):
Omar Sharif, Soroush Vosoughi, and Nikhil Singh. 2026. Improving Information Extraction with Learned Queries. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 16952–16972, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Improving Information Extraction with Learned Queries (Sharif et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.800.pdf