Comparative Analysis of Named Entity Recognition in the Dungeons and Dragons Domain

Gayashan Weerasundara, Nisansa de Silva


Abstract
Some Natural Language Processing (NLP) tasks that are in the sufficiently solved state for general domain English still struggle to attain the same level of performance in specific domains. Named Entity Recognition (NER), which aims to find and categorize entities in text is such a task met with difficulties in adapting to domain specificity. This paper compares the performance of 10 NER models on 7 adventure books from the Dungeons and Dragons (D&D) domain which is a subdomain of fantasy literature. Fantasy literature, being rich and diverse in vocabulary, poses considerable challenges for conventional NER. In this study, we use open-source Large Language Models (LLM) to annotate the named entities and character names in each number of official D&D books and evaluate the precision and distribution of each model. The paper aims to identify the challenges and opportunities for improving NER in fantasy literature. Our results show that even in the off-the-shelf configuration, Flair, Trankit, and Spacy achieve better results for identifying named entities in the D&D domain compared to their peers.
Anthology ID:
2023.ranlp-1.130
Volume:
Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing
Month:
September
Year:
2023
Address:
Varna, Bulgaria
Editors:
Ruslan Mitkov, Galia Angelova
Venue:
RANLP
SIG:
Publisher:
INCOMA Ltd., Shoumen, Bulgaria
Note:
Pages:
1225–1233
Language:
URL:
https://aclanthology.org/2023.ranlp-1.130
DOI:
Bibkey:
Cite (ACL):
Gayashan Weerasundara and Nisansa de Silva. 2023. Comparative Analysis of Named Entity Recognition in the Dungeons and Dragons Domain. In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 1225–1233, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.
Cite (Informal):
Comparative Analysis of Named Entity Recognition in the Dungeons and Dragons Domain (Weerasundara & de Silva, RANLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.ranlp-1.130.pdf