%0 Conference Proceedings %T EventBERT: Incorporating Event-based Semantics for Natural Language Understanding %A Anni, Zou %A Zhuosheng, Zhang %A Hai, Zhao %Y Sun, Maosong %Y Liu, Yang %Y Che, Wanxiang %Y Feng, Yang %Y Qiu, Xipeng %Y Rao, Gaoqi %Y Chen, Yubo %S Proceedings of the 21st Chinese National Conference on Computational Linguistics %D 2022 %8 October %I Chinese Information Processing Society of China %C Nanchang, China %G English %F anni-etal-2022-eventbert %X “Natural language understanding tasks require a comprehensive understanding of natural language and further reasoning about it, on the basis of holistic information at different levels to gain comprehensive knowledge. In recent years, pre-trained language models (PrLMs) have shown impressive performance in natural language understanding. However, they rely mainly on extracting context-sensitive statistical patterns without explicitly modeling linguistic information, such as semantic relationships entailed in natural language. In this work, we propose EventBERT, an event-based semantic representation model that takes BERT as the backbone and refines with event-based structural semantics in terms of graph convolution networks. EventBERT benefits simultaneously from rich event-based structures embodied in the graph and contextual semantics learned in pre-trained model BERT. Experimental results on the GLUE benchmark show that the proposed model consistently outperforms the baseline model.” %U https://aclanthology.org/2022.ccl-1.69 %P 774-785