Count-Based and Predictive Language Models for Exploring DeReKo

Peter Fankhauser, Marc Kupietz


Abstract
We present the use of count-based and predictive language models for exploring language use in the German Reference Corpus DeReKo. For collocation analysis along the syntagmatic axis we employ traditional association measures based on co-occurrence counts as well as predictive association measures derived from the output weights of skipgram word embeddings. For inspecting the semantic neighbourhood of words along the paradigmatic axis we visualize the high dimensional word embeddings in two dimensions using t-stochastic neighbourhood embeddings. Together, these visualizations provide a complementary, explorative approach to analysing very large corpora in addition to corpus querying. Moreover, we discuss count-based and predictive models w.r.t. scalability and maintainability in very large corpora.
Anthology ID:
2022.cmlc-1.5
Volume:
Proceedings of the Workshop on Challenges in the Management of Large Corpora (CMLC-10)
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Piotr Banski, Adrien Barbaresi, Simon Clematide, Marc Kupietz, Harald Lüngen
Venue:
CMLC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
27–31
Language:
URL:
https://aclanthology.org/2022.cmlc-1.5
DOI:
Bibkey:
Cite (ACL):
Peter Fankhauser and Marc Kupietz. 2022. Count-Based and Predictive Language Models for Exploring DeReKo. In Proceedings of the Workshop on Challenges in the Management of Large Corpora (CMLC-10), pages 27–31, Marseille, France. European Language Resources Association.
Cite (Informal):
Count-Based and Predictive Language Models for Exploring DeReKo (Fankhauser & Kupietz, CMLC 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.cmlc-1.5.pdf