Task-Agnostic Web Document Annotation with LLM-Generated Descriptors

Otto Tarkka, Erik Henriksson, Jenna Kanerva, Filip Ginter


Abstract
Web-scale text corpora provide NLP researchers and corpus linguists with access to vast quantities of language data, but efficiently identifying the subsets of a large corpus that are relevant to a particular research aim remains a significant challenge. Existing approaches, including keyword search and task-specific classifiers, are often either too restrictive or depend on costly retraining for each new research question. We explore and evaluate the feasibility of a task-agnostic, LLM-driven descriptor schema for annotating web documents, using LLMs to generate natural-language labels that capture stylistic, tonal, topical, and structural properties of texts. Unlike prior work that emphasizes broad categorization or quality filtering, our approach yields fine-grained and interpretable descriptors designed to support flexible downstream applications. We evaluate the approach from multiple angles and show that descriptor-based annotation can enable flexible document retrieval while also exposing challenges related to schema harmonization and descriptor specificity.
Anthology ID:
2026.wac-1.24
Volume:
Proceedings of the 13th Web-as-Corpus Workshop
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pedro Ortiz Suarez, Laurie Burchell, Nikola Ljubešić, Yves Scherrer, Veronika Laippala, Vuk Dinić, Thom Vaughan
Venues:
WAC | WS
SIG:
SIGWAC
Publisher:
Association for Computational Linguistics
Note:
Pages:
252–281
Language:
URL:
https://aclanthology.org/2026.wac-1.24/
DOI:
Bibkey:
Cite (ACL):
Otto Tarkka, Erik Henriksson, Jenna Kanerva, and Filip Ginter. 2026. Task-Agnostic Web Document Annotation with LLM-Generated Descriptors. In Proceedings of the 13th Web-as-Corpus Workshop, pages 252–281, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Task-Agnostic Web Document Annotation with LLM-Generated Descriptors (Tarkka et al., WAC 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.wac-1.24.pdf