K-hop neighbourhood regularization for few-shot learning on graphs: A case study of text classification

Niels van der Heijden, Ekaterina Shutova, Helen Yannakoudakis


Abstract
We present FewShotTextGCN, a novel method designed to effectively utilize the properties of word-document graphs for improved learning in low-resource settings. We introduce K-hop Neighbourhood Regularization, a regularizer for heterogeneous graphs, and show that it stabilizes and improves learning when only a few training samples are available. We furthermore propose a simplification in the graph-construction method, which results in a graph that is ∼7 times less dense and yields better performance in little-resource settings while performing on par with the state of the art in high-resource settings. Finally, we introduce a new variant of Adaptive Pseudo-Labeling tailored for word-document graphs. When using as little as 20 samples for training, we outperform a strong TextGCN baseline with 17% in absolute accuracy on average over eight languages. We demonstrate that our method can be applied to document classification without any language model pretraining on a wide range of typologically diverse languages while performing on par with large pretrained language models.
Anthology ID:
2023.eacl-main.85
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1187–1200
Language:
URL:
https://aclanthology.org/2023.eacl-main.85
DOI:
10.18653/v1/2023.eacl-main.85
Bibkey:
Cite (ACL):
Niels van der Heijden, Ekaterina Shutova, and Helen Yannakoudakis. 2023. K-hop neighbourhood regularization for few-shot learning on graphs: A case study of text classification. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1187–1200, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
K-hop neighbourhood regularization for few-shot learning on graphs: A case study of text classification (van der Heijden et al., EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-main.85.pdf
Video:
 https://aclanthology.org/2023.eacl-main.85.mp4