On the Concept of Resource-Efficiency in NLP

Luise Dürlich, Evangelia Gogoulou, Joakim Nivre


Abstract
Resource-efficiency is a growing concern in the NLP community. But what are the resources we care about and why? How do we measure efficiency in a way that is reliable and relevant? And how do we balance efficiency and other important concerns? Based on a review of the emerging literature on the subject, we discuss different ways of conceptualizing efficiency in terms of product and cost, using a simple case study on fine-tuning and knowledge distillation for illustration. We propose a novel metric of amortized efficiency that is better suited for life-cycle analysis than existing metrics.
Anthology ID:
2023.nodalida-1.15
Volume:
Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)
Month:
May
Year:
2023
Address:
Tórshavn, Faroe Islands
Editors:
Tanel Alumäe, Mark Fishel
Venue:
NoDaLiDa
SIG:
Publisher:
University of Tartu Library
Note:
Pages:
135–145
Language:
URL:
https://aclanthology.org/2023.nodalida-1.15
DOI:
Bibkey:
Cite (ACL):
Luise Dürlich, Evangelia Gogoulou, and Joakim Nivre. 2023. On the Concept of Resource-Efficiency in NLP. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 135–145, Tórshavn, Faroe Islands. University of Tartu Library.
Cite (Informal):
On the Concept of Resource-Efficiency in NLP (Dürlich et al., NoDaLiDa 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.nodalida-1.15.pdf