A Hybrid Knowledge and Transformer-Based Model for Event Detection with Automatic Self-Attention Threshold, Layer and Head Selection

Thierry Desot, Orphee De Clercq, Veronique Hoste


Abstract
Event and argument role detection are frequently conceived as separate tasks. In this work we conceive both processes as one taskin a hybrid event detection approach. Its main component is based on automatic keyword extraction (AKE) using the self-attention mechanism of a BERT transformer model. As a bottleneck for AKE is defining the threshold of the attention values, we propose a novel method for automatic self-attention thresholdselection. It is fueled by core event information, or simply the verb and its arguments as the backbone of an event. These are outputted by a knowledge-based syntactic parser. In a secondstep the event core is enriched with other semantically salient words provided by the transformer model. Furthermore, we propose an automatic self-attention layer and head selectionmechanism, by analyzing which self-attention cells in the BERT transformer contribute most to the hybrid event detection and which linguistic tasks they represent. This approach was integrated in a pipeline event extraction approachand outperforms three state of the art multi-task event extraction methods.
Anthology ID:
2022.case-1.4
Volume:
Proceedings of the 5th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE)
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates (Hybrid)
Editors:
Ali Hürriyetoğlu, Hristo Tanev, Vanni Zavarella, Erdem Yörük
Venue:
CASE
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
21–31
Language:
URL:
https://aclanthology.org/2022.case-1.4
DOI:
10.18653/v1/2022.case-1.4
Bibkey:
Cite (ACL):
Thierry Desot, Orphee De Clercq, and Veronique Hoste. 2022. A Hybrid Knowledge and Transformer-Based Model for Event Detection with Automatic Self-Attention Threshold, Layer and Head Selection. In Proceedings of the 5th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE), pages 21–31, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Cite (Informal):
A Hybrid Knowledge and Transformer-Based Model for Event Detection with Automatic Self-Attention Threshold, Layer and Head Selection (Desot et al., CASE 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.case-1.4.pdf
Video:
 https://aclanthology.org/2022.case-1.4.mp4