SCALE: A Scalable Language Engineering Toolkit

Joris Pelemans, Lyan Verwimp, Kris Demuynck, Hugo Van hamme, Patrick Wambacq


Abstract
In this paper we present SCALE, a new Python toolkit that contains two extensions to n-gram language models. The first extension is a novel technique to model compound words called Semantic Head Mapping (SHM). The second extension, Bag-of-Words Language Modeling (BagLM), bundles popular models such as Latent Semantic Analysis and Continuous Skip-grams. Both extensions scale to large data and allow the integration into first-pass ASR decoding. The toolkit is open source, includes working examples and can be found on http://github.com/jorispelemans/scale.
Anthology ID:
L16-1612
Volume:
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
Month:
May
Year:
2016
Address:
Portorož, Slovenia
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
3868–3871
Language:
URL:
https://aclanthology.org/L16-1612
DOI:
Bibkey:
Cite (ACL):
Joris Pelemans, Lyan Verwimp, Kris Demuynck, Hugo Van hamme, and Patrick Wambacq. 2016. SCALE: A Scalable Language Engineering Toolkit. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 3868–3871, Portorož, Slovenia. European Language Resources Association (ELRA).
Cite (Informal):
SCALE: A Scalable Language Engineering Toolkit (Pelemans et al., LREC 2016)
Copy Citation:
PDF:
https://aclanthology.org/L16-1612.pdf
Code
 jorispelemans/scale