ArmyForRE at KnowledgeGraphEval 2026: Typed Entity Markers and Imbalance-Aware Training for Arabic Relation Extraction

Milyun Ni’ma Shoumi, Arie Rachmad Syulistyo, Aulia Resty Azizah, Savero Athallah Hardiana Putra


Abstract
In this paper, we present a relation extraction system developed for Subtask 2 of KnowledgeGraphEval 2026. The system fine-tunes a single ARBERTv2 encoder on inputs with typed entity markers that expose entity boundaries and types, also classifies each entity pair from the encoder’s classification token ([CLS]) state concatenated with the states at the subject and object start markers. It is trained over the full 41-label space, including no-relation, with focal loss, class-balanced sampling, label smoothing, and no-relation down-weighting to counter severe class imbalance. Our submission ranked third of seven teams with a micro-F1 of 0.9332, 0.23 points above the official baseline, which fine-tunes the same encoder. We also report an analysis of seed-dependent behaviour on fine-grained geopolitical relations from our earlier development experiments and discuss the limitations of our evaluation.
Anthology ID:
2026.arabicnlp-sharedtasks.158
Volume:
Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Muhammad Abdul-Mageed, Tamer Elsayed, Houda Bouamor, Samhaa El-Beltagy, Hend Al-Khalifa, Ahmed Abdelali, AbdelRahim Elmadany, Abdellah El Mekki, Kareem Darwish, Sakhar Alkhereyf, Hamzah Luqman, Salam Khalifa, Samar Magdy, Zaid Alyafeai, Injy Hamed, Wajdi Zaghouani, Imed Zitouni
Venue:
ArabicNLP
SIG:
SIGARAB
Publisher:
Association for Computational Linguistics
Note:
Pages:
1255–1260
Language:
URL:
https://aclanthology.org/2026.arabicnlp-sharedtasks.158/
DOI:
Bibkey:
Cite (ACL):
Milyun Ni’ma Shoumi, Arie Rachmad Syulistyo, Aulia Resty Azizah, and Savero Athallah Hardiana Putra. 2026. ArmyForRE at KnowledgeGraphEval 2026: Typed Entity Markers and Imbalance-Aware Training for Arabic Relation Extraction. In Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks, pages 1255–1260, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
ArmyForRE at KnowledgeGraphEval 2026: Typed Entity Markers and Imbalance-Aware Training for Arabic Relation Extraction (Shoumi et al., ArabicNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.arabicnlp-sharedtasks.158.pdf