Large-scale web corpora are valuable resources for natural language processing (NLP), linguistic research, and language model training, particularly for digitally low-resource languages such as Persian. However, web-derived text often contains document-level noise, even after quality-filtering pipelines. We present HPLT3 Persian Refined (HPLT3-PerRef), a refined Persian web corpus containing 105.9 billion tokens and 102.37 million documents, obtained by applying a supervised quality-filtering pipeline to the Persian subset of HPLT3. We manually annotated 6,144 Persian web documents and trained a multilingual transformer-based classifier to distinguish high- and low-quality documents. The classifier removed approximately 20% of the original corpus while preserving broad domain coverage and complements HPLT3’s heuristic-based Web Document Score (WDS) by identifying additional low-quality documents. We publicly release the corpus, the trained classifier, and the complete source code.

Web registers, the situationally defined varieties of online text, are almost always annotated at document level. A document, however, may contain multiple such varieties. We label 80,000 documents from the HPLT web corpus at segment level in seven languages using the open-weight Gemma 4, reaching Cohen’s κ=0.73 against 1,878 human-annotated segments, comparable to previously reported document-level agreement. We find that most documents have two or more registers, and that they recur in similar positions and combinations across the studied languages. Yet on average, a single register accounts for 73% of the labeled text, and a quarter to a third of segments receive no label. Document-level labels are therefore mostly correct but incomplete. Multi-dimensional analysis shows that the segment-level labels correspond to linguistic differences. In all seven languages, registers are ordered the same way, from informational to involved text.

Corpus filtering strategies have evolved steadily, leading to increasingly effective data selection techniques that improve downstream LLM performance. However, while most filters focus on knowledge-rich and educational content, their effect on the linguistic and thematic composition of the documents in pre-training corpora remains underexplored. In this paper, we investigate how educational filtering affects both linguistic and thematic content by combining Propella-1 educational value annotations and the FinePDFs-Edu filter with the MultiCORE register (genre) framework. Focusing on the Finnish subset of the HPLT 4.0 corpus, we examine how educational filtering changes the distribution of registers and how educational value relates to register categories. By studying machine translated documents with high educational value we show that educational filters focus on content over textual quality. We further analyze changes in the thematic composition using Propella-1 business sector labels, examining whether educational filtering shifts the representation of different content domains. Finally, to assess the impact of filtering, we trained three small language models and evaluated their downstream performance on benchmark tasks. We find that educational filters disagree in their assessment of document educational value, show similar preferences for linguistic registers, but differ in the thematic content they select, resulting in different downstream performance across task types.

The development of high-quality multilingual models depends on well-curated multilingual datasets, which are typically obtained through filtering large web-crawled corpora. Such filtering approaches must be effective across a wide range of languages in order to improve model quality proportionally across the entire linguistic space. In recent years multiple Natural Language Processing (NLP) approaches for data processing and Large Language Model (LLM) training were developed. However, the most significant progress has been made for rich-resource languages such as English and Chinese, leaving multilingual methods largely underexplored. In this work, we present data selection strategies based on Internet registers (genres) and Web Document Score (WDS). We study 15 European languages utilizing HPLT 3.0 datasets. We train and evaluate four small generative language models per language, and compare how specific registers and WDS affect model training in separate and combined approaches. We use five common evaluation tasks covering models’ linguistic, knowledge-based, and reasoning skills. We analyze the impact of the studied strategies on each language, their average performance, and show how combining register-based selection with WDS improves evaluation performance. These results indicate that both registers and WDS can generalize across multiple languages, offering a new path towards multilingual data filtering.

Open web-scale pre-training corpora remain concentrated in English, limiting multilingual LLM development. We introduce MultiSynt/MT, an open synthetic parallel corpus with approximately 4.8 trillion target-language tokens across 36 European languages, produced by translating 100 billion high-quality Nemotron-CC tokens with Tower+ and OPUS-MT/HPLT-MT systems. For many medium- and lower-resource European languages, this is the largest openly available pre-training resource. On a broad multilingual benchmark suite, reference LLMs trained on MultiSynt/MT reach the final score of HPLT 2.0, a native-data baseline, using roughly 72% fewer pre-training tokens, and outperform it by approximately 15% relative at a matched 100B-token training budget. Our analyses also identify evaluation blind spots: standard multiple-choice benchmarks miss translation-quality differences that a fluency-sensitive LLM-as-judge evaluation cleanly recovers on the trained LLMs (with no fluency deficit in MultiSynt itself), and Norwegian idiomatic and culturally grounded tasks remain better served by native data. We release the corpus, including row-aligned translations from multiple systems, to support controlled research on multilingual pre-training data and evaluation.