One format to rule them all – The emtsv pipeline for Hungarian

Balázs Indig, Bálint Sass, Eszter Simon, Iván Mittelholcz, Noémi Vadász, Márton Makrai


Abstract
We present a more efficient version of the e-magyar NLP pipeline for Hungarian called emtsv. It integrates Hungarian NLP tools in a framework whose individual modules can be developed or replaced independently and allows new ones to be added. The design also allows convenient investigation and manual correction of the data flow from one module to another. The improvements we publish include effective communication between the modules and support of the use of individual modules both in the chain and standing alone. Our goals are accomplished using extended tsv (tab separated values) files, a simple, uniform, generic and self-documenting input/output format. Our vision is maintaining the system for a long time and making it easier for external developers to fit their own modules into the system, thus sharing existing competencies in the field of processing Hungarian, a mid-resourced language. The source code is available under LGPL 3.0 license at https://github.com/dlt-rilmta/emtsv .
Anthology ID:
W19-4018
Volume:
Proceedings of the 13th Linguistic Annotation Workshop
Month:
August
Year:
2019
Address:
Florence, Italy
Editors:
Annemarie Friedrich, Deniz Zeyrek, Jet Hoek
Venue:
LAW
SIG:
SIGANN
Publisher:
Association for Computational Linguistics
Note:
Pages:
155–165
Language:
URL:
https://aclanthology.org/W19-4018
DOI:
10.18653/v1/W19-4018
Bibkey:
Cite (ACL):
Balázs Indig, Bálint Sass, Eszter Simon, Iván Mittelholcz, Noémi Vadász, and Márton Makrai. 2019. One format to rule them all – The emtsv pipeline for Hungarian. In Proceedings of the 13th Linguistic Annotation Workshop, pages 155–165, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
One format to rule them all – The emtsv pipeline for Hungarian (Indig et al., LAW 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-4018.pdf
Code
 dlt-rilmta/emtsv