Knodle: Modular Weakly Supervised Learning with PyTorch

Anastasiia Sedova, Andreas Stephan, Marina Speranskaya, Benjamin Roth


Abstract
Strategies for improving the training and prediction quality of weakly supervised machine learning models vary in how much they are tailored to a specific task or integrated with a specific model architecture. In this work, we introduce Knodle, a software framework that treats weak data annotations, deep learning models, and methods for improving weakly supervised training as separate, modular components. This modularization gives the training process access to fine-grained information such as data set characteristics, matches of heuristic rules, or elements of the deep learning model ultimately used for prediction. Hence, our framework can encompass a wide range of training methods for improving weak supervision, ranging from methods that only look at correlations of rules and output classes (independently of the machine learning model trained with the resulting labels), to those that harness the interplay of neural networks and weakly labeled data. We illustrate the benchmarking potential of the framework with a performance comparison of several reference implementations on a selection of datasets that are already available in Knodle.
Anthology ID:
2021.repl4nlp-1.12
Volume:
Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021)
Month:
August
Year:
2021
Address:
Online
Editors:
Anna Rogers, Iacer Calixto, Ivan Vulić, Naomi Saphra, Nora Kassner, Oana-Maria Camburu, Trapit Bansal, Vered Shwartz
Venue:
RepL4NLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
100–111
Language:
URL:
https://aclanthology.org/2021.repl4nlp-1.12
DOI:
10.18653/v1/2021.repl4nlp-1.12
Bibkey:
Cite (ACL):
Anastasiia Sedova, Andreas Stephan, Marina Speranskaya, and Benjamin Roth. 2021. Knodle: Modular Weakly Supervised Learning with PyTorch. In Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021), pages 100–111, Online. Association for Computational Linguistics.
Cite (Informal):
Knodle: Modular Weakly Supervised Learning with PyTorch (Sedova et al., RepL4NLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.repl4nlp-1.12.pdf
Video:
 https://aclanthology.org/2021.repl4nlp-1.12.mp4
Code
 knodle/knodle
Data
IMDb Movie ReviewsSMS Spam Collection Data SetTACRED