Tuomas Lundberg
Author directory2026
Segment-Level Register Analysis of the Multilingual Web
Erik Henriksson | Alireza Razzaghi | Tomasz Galica | Jiaqi Feng Guo | Anastasia Zhi-Ling Dong | Urho Kunnas | Erofili Psaltaki | Tuomas Lundberg | Antti Kanner | Veronika Laippala
Proceedings of the 13th Web-as-Corpus Workshop
Erik Henriksson | Alireza Razzaghi | Tomasz Galica | Jiaqi Feng Guo | Anastasia Zhi-Ling Dong | Urho Kunnas | Erofili Psaltaki | Tuomas Lundberg | Antti Kanner | Veronika Laippala
Proceedings of the 13th Web-as-Corpus Workshop
Web registers, the situationally defined varieties of online text, are almost always annotated at document level. A document, however, may contain multiple such varieties. We label 80,000 documents from the HPLT web corpus at segment level in seven languages using the open-weight Gemma 4, reaching Cohen’s κ=0.73 against 1,878 human-annotated segments, comparable to previously reported document-level agreement. We find that most documents have two or more registers, and that they recur in similar positions and combinations across the studied languages. Yet on average, a single register accounts for 73% of the labeled text, and a quarter to a third of segments receive no label. Document-level labels are therefore mostly correct but incomplete. Multi-dimensional analysis shows that the segment-level labels correspond to linguistic differences. In all seven languages, registers are ordered the same way, from informational to involved text.
Towards More Reliable Web Metadata: Do Register Classifiers Learn Register or Genre Features?
Antti Kanner | Tuomas Lundberg | Erik Henriksson | Veronika Laippala
Proceedings of the 13th Web-as-Corpus Workshop
Antti Kanner | Tuomas Lundberg | Erik Henriksson | Veronika Laippala
Proceedings of the 13th Web-as-Corpus Workshop
Transformer-based web register classifiers are widely used to provide metadata for web crawls and to focus web-data sampling on specific register classes, such as news articles or advertisements. However, it remains unclear what these classifiers actually learn: do they detect pervasive linguistic features characteristic of registers, or do they rely on structural cues more indicative of genre? We examine this question through two hypotheses: the text-macrostructure hypothesis proposes that classifiers rely on structural features more typical of genres, whereas the feature-density hypothesis predicts that they identify typically register-associated features. Using English, Finnish, and French data from HPLT 3.0, we segment documents into thirds and compare classifier predictions for full documents and segments through five complementary analyses: main-label agreement, cosine similarity, confusion profiles, SACX keyword-weight correlations, and SACX keyword stability. As a more direct control, we compare predictions for a sample of untruncated documents with predictions for the same documents after their sentence order has been randomised. Top-label assignments and prediction vectors remain highly similar after shuffling, although the positional gradients are attenuated, indicating that sentence order contributes to classifier behaviour without determining its broader prediction profile. Overall, the results are more consistent with the feature-density hypothesis, although both hypotheses retain explanatory value.
Register Mixing Is the Norm on the Web
Erik Henriksson | Alireza Razzaghi | Tuomas Lundberg | Antti Kanner | Veronika Laippala
Proceedings of the 6th International Conference on Natural Language Processing for the Digital Humanities
Erik Henriksson | Alireza Razzaghi | Tuomas Lundberg | Antti Kanner | Veronika Laippala
Proceedings of the 6th International Conference on Natural Language Processing for the Digital Humanities
Nearly all studies on web registers—online text varieties associated with characteristic social contexts and linguistic features—use full documents as the unit of analysis. However, web documents often contain sections in different registers. A cooking blog, for instance, may combine personal storytelling, recipe instructions, user comments, and promotional text within a single URL. This internal variation raises doubts about the validity of document level register labeling. In this paper, we propose an LLM-based approach that identifies register homogeneous segments within documents and apply it to a 10,000-document English sample from HPLT 3.0. We show that segmentation addresses persistent problems in register analysis, including low inter-annotator agreement and category fuzziness. Strikingly, it also reveals that most web documents contain more than one register, making register mixing the norm rather than the exception on the web.