Designing LLM Agents for User-Centered Language Service Selection

Ryoichiro Ogawa, Donghui Lin, Fumito Uwano


Abstract
With the rapid expansion of language resources and services across repositories and platforms, users face an overwhelming number of options. While this diversity promises flexibility, non-experts struggle to compose appropriate resource pipelines and select services that satisfy both functional and non-functional requirements. We propose a user-centered framework of LLM agents that interprets natural-language requests and performs end-to-end language service selection. The agents extract functional requirements to form coherent task compositions and select suitable language services for each component by interpreting non-functional quality aspects embedded in contextual cues. To ensure reliable and explainable decisions, we employ a four-step structured reasoning procedure that combines Few-Shot exemplars and Chain-of-Thought reasoning: extracting functional requirements, inducing non-functional evaluation axes, applying these axes as constraints in candidate retrieval, and determining a final composition. We construct a benchmark dataset pairing diverse user requests with standardized language service profiles containing metadata and quality indicators, and evaluate our framework against representative prompting-based baselines. Results show consistent gains in Precision, Recall, and F1-score, demonstrating improved capture of both functional intent and quality preferences. These findings demonstrate that structured LLM agents can bridge natural-language user intents and language service configurations, enabling end-to-end selection and composition in a transparent and user-centered manner.
Anthology ID:
2026.lrec-1.42
Volume:
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Month:
May
Year:
2026
Address:
Palma de Mallorca, Spain
Editors:
Stelios Piperidis, Núria Bel, Henk van den Heuvel, Nancy Ide, Simon Krek, Antonio Toral
Venue:
LREC
SIG:
Publisher:
ELRA Language Resource Association
Note:
Pages:
599–608
Language:
External URL:
https://lrec.elra.info/lrec2026-main-042
DOI:
10.63317/59qnh9w46bv6
Bibkey:
Cite (ACL):
Ryoichiro Ogawa, Donghui Lin, and Fumito Uwano. 2026. Designing LLM Agents for User-Centered Language Service Selection. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 599–608, Palma de Mallorca, Spain. ELRA Language Resource Association.
Cite (Informal):
Designing LLM Agents for User-Centered Language Service Selection (Ogawa et al., LREC 2026)
Copy Citation: