Enhanced Language Representation with Label Knowledge for Span Extraction

Pan Yang, Xin Cong, Zhenyu Sun, Xingwu Liu


Abstract
Span extraction, aiming to extract text spans (such as words or phrases) from plain text, is a fundamental process in Information Extraction. Recent works introduce the label knowledge to enhance the text representation by formalizing the span extraction task into a question answering problem (QA Formalization), which achieves state-of-the-art performance. However, such a QA Formalization does not fully exploit the label knowledge and causes a dramatic decrease in efficiency of training/inference. To address those problems, we introduce a fresh paradigm to integrate label knowledge and further propose a novel model to explicitly and efficiently integrate label knowledge into text representations. Specifically, it encodes texts and label annotations independently and then integrates label knowledge into text representation with an elaborate-designed semantics fusion module. We conduct extensive experiments on three typical span extraction tasks: flat NER, nested NER, and event detection. The empirical results show that 1) our model achieves a new state-of-the-art performance on four benchmarks, and 2) reduces training time and inference time by 76% and 77% on average, respectively, compared with the QA Formalization paradigm.
Anthology ID:
2021.emnlp-main.379
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4623–4635
Language:
URL:
https://aclanthology.org/2021.emnlp-main.379
DOI:
10.18653/v1/2021.emnlp-main.379
Bibkey:
Cite (ACL):
Pan Yang, Xin Cong, Zhenyu Sun, and Xingwu Liu. 2021. Enhanced Language Representation with Label Knowledge for Span Extraction. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4623–4635, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Enhanced Language Representation with Label Knowledge for Span Extraction (Yang et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.379.pdf
Video:
 https://aclanthology.org/2021.emnlp-main.379.mp4
Code
 akeepers/lear