Urho Kunnas

Author directory

2026

Web registers, the situationally defined varieties of online text, are almost always annotated at document level. A document, however, may contain multiple such varieties. We label 80,000 documents from the HPLT web corpus at segment level in seven languages using the open-weight Gemma 4, reaching Cohen’s κ=0.73 against 1,878 human-annotated segments, comparable to previously reported document-level agreement. We find that most documents have two or more registers, and that they recur in similar positions and combinations across the studied languages. Yet on average, a single register accounts for 73% of the labeled text, and a quarter to a third of segments receive no label. Document-level labels are therefore mostly correct but incomplete. Multi-dimensional analysis shows that the segment-level labels correspond to linguistic differences. In all seven languages, registers are ordered the same way, from informational to involved text.
Corpus filtering strategies have evolved steadily, leading to increasingly effective data selection techniques that improve downstream LLM performance. However, while most filters focus on knowledge-rich and educational content, their effect on the linguistic and thematic composition of the documents in pre-training corpora remains underexplored. In this paper, we investigate how educational filtering affects both linguistic and thematic content by combining Propella-1 educational value annotations and the FinePDFs-Edu filter with the MultiCORE register (genre) framework. Focusing on the Finnish subset of the HPLT 4.0 corpus, we examine how educational filtering changes the distribution of registers and how educational value relates to register categories. By studying machine translated documents with high educational value we show that educational filters focus on content over textual quality. We further analyze changes in the thematic composition using Propella-1 business sector labels, examining whether educational filtering shifts the representation of different content domains. Finally, to assess the impact of filtering, we trained three small language models and evaluated their downstream performance on benchmark tasks. We find that educational filters disagree in their assessment of document educational value, show similar preferences for linguistic registers, but differ in the thematic content they select, resulting in different downstream performance across task types.