Alaa Aljabari
2024
Event-Arguments Extraction Corpus and Modeling using BERT for Arabic
Alaa Aljabari
|
Lina Duaibes
|
Mustafa Jarrar
|
Mohammed Khalilia
Proceedings of The Second Arabic Natural Language Processing Conference
Event-argument extraction is a challenging task, particularly in Arabic due to sparse linguistic resources. To fill this gap, we introduce the corpus (550k tokens) as an extension of Wojood, enriched with event-argument annotations. We used three types of event arguments: agent, location, and date, which we annotated as relation types. Our inter-annotator agreement evaluation resulted in 82.23% Kappa score and 87.2% F1-score. Additionally, we propose a novel method for event relation extraction using BERT, in which we treat the task as text entailment. This method achieves an F1-score of 94.01%.To further evaluate the generalization of our proposed method, we collected and annotated another out-of-domain corpus (about 80k tokens) called and used it as a second test set, on which our approach achieved promising results (83.59% F1-score). Last but not least, we propose an end-to-end system for event-arguments extraction. This system is implemented as part of SinaTools, and both corpora are publicly available at https://sina.birzeit.edu/wojood
ArabicNLU 2024: The First Arabic Natural Language Understanding Shared Task
Mohammed Khalilia
|
Sanad Malaysha
|
Reem Suwaileh
|
Mustafa Jarrar
|
Alaa Aljabari
|
Tamer Elsayed
|
Imed Zitouni
Proceedings of The Second Arabic Natural Language Processing Conference
This paper presents an overview of the Arabic Natural Language Understanding (ArabicNLU 2024) shared task, focusing on two subtasks: Word Sense Disambiguation (WSD) and Location Mention Disambiguation (LMD). The task aimed to evaluate the ability of automated systems to resolve word ambiguity and identify locations mentioned in Arabic text. We provided participants with novel datasets, including a sense-annotated corpus for WSD, called SALMA with approximately 34k annotated tokens, and the dataset with 3,893 annotations and 763 unique location mentions. These are challenging tasks. Out of the 38 registered teams, only three teams participated in the final evaluation phase, with the highest accuracy being 77.8% for WSD and 95.0% for LMD. The shared task not only facilitated the evaluation and comparison of different techniques, but also provided valuable insights and resources for the continued advancement of Arabic NLU technologies.
Search
Co-authors
- Mustafa Jarrar 2
- Mohammed Khalilia 2
- Lina Duaibes 1
- Sanad Malaysha 1
- Reem Suwaileh 1
- show all...