Jonathan Geuter
Author directory2026
Thinking at the Right Size: Amortized Distillation Across Post-Trained LLMs
Yan Zhou | Sara Kangaslahti | Jonathan Geuter | Nihal V. Nayak | Marco Fumero | Francesco Locatello | David Alvarez-Melis
Findings of the Association for Computational Linguistics: EMNLP 2026
Yan Zhou | Sara Kangaslahti | Jonathan Geuter | Nihal V. Nayak | Marco Fumero | Francesco Locatello | David Alvarez-Melis
Findings of the Association for Computational Linguistics: EMNLP 2026
Practical deployment of large language models (LLMs) requires families of post-trained variants—instruction-tuned, reasoning-tuned, and chat-style models—each at multiple sizes to meet diverse latency and memory budgets. Producing each (variant, size) pair independently is prohibitive, so model families typically span only a handful of coarse-grained sizes per post-trained variant. Boomerang distillation (Kangaslahti et al., 2026) reduces this cost along the size axis for base models. Through model size interpolation, it constructs models of intermediate sizes from a single teacher-student pair without additional training. However, it still treats each post-trained variant as a separate object of optimization. We introduce ADAPT—Amortized Distillation Across Post-Trained LLMs—a framework for amortizing distillation across both axes of a model family: size and post-training variant, producing L × K models for L interpolated sizes across K post-trained variants with a single distillation run. ADAPT combines two components. First, a two-phase distillation procedure constructs post-trained students through pre-training alignment and supervised fine-tuning distillation, enabling smooth size–performance interpolation on generation and reasoning tasks. Second, weight-delta initialization approximates this construction across post-trained variants by transferring the distillation-induced weight change from the base model to students initialized from different post-trained variants. The resulting continuum of interpolated models also enables adaptive model-size selection at inference time, improving the compute–accuracy trade-off for long-form reasoning tasks.
2023
Jina Embeddings: A Novel Set of High-Performance Sentence Embedding Models
Michael Günther | Louis Milliken | Jonathan Geuter | Georgios Mastrapas | Bo Wang | Han Xiao
Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023)
Michael Günther | Louis Milliken | Jonathan Geuter | Georgios Mastrapas | Bo Wang | Han Xiao
Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023)
Jina Embeddings constitutes a set of high-performance sentence embedding models adept at translating textual inputs into numerical representations, capturing the semantics of the text. These models excel in applications like dense retrieval and semantic textual similarity. This paper details the development of Jina Embeddings, starting with the creation of high-quality pairwise and triplet datasets.It underlines the crucial role of data cleaning in dataset preparation, offers in-depth insights into the model training process, and concludes with a comprehensive performance evaluation using the Massive Text Embedding Benchmark (MTEB). Furthermore, to increase the model’s awareness of grammatical negation, we construct a novel training and evaluation dataset of negated and non-negated statements, which we make publicly available to the community.