The Construction and Evaluation of the LEAFTOP Dataset of Automatically Extracted Nouns in 1480 Languages

Gregory Baker, Diego Molla


Abstract
The LEAFTOP (language extracted automatically from thousands of passages) dataset consists of nouns that appear in multiple places in the four gospels of the New Testament. We use a naive approach — probabilistic inference — to identify likely translations in 1480 other languages. We evaluate this process and find that it provides lexiconaries with accuracy from 42% (Korafe) to 99% (Runyankole), averaging 72% correct across evaluated languages. The process translates up to 161 distinct lemmas from Koine Greek (average 159). We identify nouns which appear to be easy and hard to translate, language families where this technique works, and future possible improvements and extensions. The claims to novelty are: the use of a Koine Greek New Testament as the source language; using a fully-annotated manually-created grammatically parse of the source text; a custom scraper for texts in the target languages; a new metric for language similarity; a novel strategy for evaluation on low-resource languages.
Anthology ID:
2022.lrec-1.536
Volume:
Proceedings of the Thirteenth Language Resources and Evaluation Conference
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
5021–5028
Language:
URL:
https://aclanthology.org/2022.lrec-1.536
DOI:
Bibkey:
Cite (ACL):
Gregory Baker and Diego Molla. 2022. The Construction and Evaluation of the LEAFTOP Dataset of Automatically Extracted Nouns in 1480 Languages. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5021–5028, Marseille, France. European Language Resources Association.
Cite (Informal):
The Construction and Evaluation of the LEAFTOP Dataset of Automatically Extracted Nouns in 1480 Languages (Baker & Molla, LREC 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.lrec-1.536.pdf
Data
Panlex