Energy-Aware Benchmarking of Generative Information Extraction

Andrei Politov, Iryna Vergunova, Rene Jäkel, Michael Färber


Abstract
Using Large Language Models (LLMs) for Information Extraction (IE) inherently involves transforming diverse textual inputs into strictly formatted outputs. Guaranteeing this structural validity introduces a hidden energy tax driven by constrained decoding mechanisms, serialisation formats, and underlying serving configurations. Consequently, standard token-level metrics fail to capture the true energy cost of generative IE, which also cannot be predicted by model size alone. To address this, we introduce the first dedicated benchmark evaluating the energy-accuracy trade-offs of generative IE. Our framework establishes task-aligned energy metrics, including Energy per True Positive (J/TP) and the F1-Energy Ratio (F1/kJ). Through comprehensive ablation across architectures, decoding regimes, and serving-layer limits, we demonstrate that efficiency is governed by complex parameter interactions. We translate these findings into actionable configuration guidelines, equipping practitioners to maximise predictive accuracy, optimise energy yield, or achieve a sustainable Pareto balance.
Anthology ID:
2026.findings-emnlp.2002
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
42668–42691
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2002/
DOI:
Bibkey:
Cite (ACL):
Andrei Politov, Iryna Vergunova, Rene Jäkel, and Michael Färber. 2026. Energy-Aware Benchmarking of Generative Information Extraction. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 42668–42691, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Energy-Aware Benchmarking of Generative Information Extraction (Politov et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2002.pdf