A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation

Tianxiang Sun, Xiangyang Liu, Wei Zhu, Zhichao Geng, Lingling Wu, Yilong He, Yuan Ni, Guotong Xie, Xuanjing Huang, Xipeng Qiu


Abstract
Early exiting allows instances to exit at different layers according to the estimation of difficulty. Previous works usually adopt heuristic metrics such as the entropy of internal outputs to measure instance difficulty, which suffers from generalization and threshold-tuning. In contrast, learning to exit, or learning to predict instance difficulty is a more appealing way. Though some effort has been devoted to employing such “learn-to-exit” modules, it is still unknown whether and how well the instance difficulty can be learned. As a response, we first conduct experiments on the learnability of instance difficulty, which demonstrates that modern neural models perform poorly on predicting instance difficulty. Based on this observation, we propose a simple-yet-effective Hash-based Early Exiting approach HashEE) that replaces the learn-to-exit modules with hash functions to assign each token to a fixed exiting layer. Different from previous methods, HashEE requires no internal classifiers nor extra parameters, and therefore is more efficient. HashEE can be used in various tasks (including language understanding and generation) and model architectures such as seq2seq models. Experimental results on classification, regression, and generation tasks demonstrate that HashEE can achieve higher performance with fewer FLOPs and inference time compared with previous state-of-the-art early exiting methods.
Anthology ID:
2022.findings-acl.189
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2409–2421
Language:
URL:
https://aclanthology.org/2022.findings-acl.189
DOI:
10.18653/v1/2022.findings-acl.189
Bibkey:
Cite (ACL):
Tianxiang Sun, Xiangyang Liu, Wei Zhu, Zhichao Geng, Lingling Wu, Yilong He, Yuan Ni, Guotong Xie, Xuanjing Huang, and Xipeng Qiu. 2022. A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2409–2421, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation (Sun et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.189.pdf
Code
 txsun1997/hashee
Data
IMDb Movie ReviewsMRPCSNLISSTSST-2