Improved Personalized Typing Predictions via Long Left Context

Soufia Bahmani, Keith Vertanen


Abstract
We study how to improve predictive typing via per-user personalization of a pretrained large language model (LLM). We evaluate personalization in a time-ordered setting, using each user’s earlier messages for adaptation as we try to predict their next message. We compare: (1) personalization versus a non-personalized base model, (2) differing amounts of left context, (3) one-time adaptation versus making small periodic updates, and (4) full fine-tuning versus parameter-efficient fine-tuning. We find both static and personalized LLMs significantly outperform an n-gram language model. This includes when the n-gram was mixed with a user model learned via prediction by partial matching (PPM). Using long recent user-specific context provided the largest personalization gains: conditioning on up to 1024 previous characters reduced the base model’s perplexity by 18%. We found fine-tuning model weights provided only a small additional gain.
Anthology ID:
2026.winlp-main.11
Volume:
Proceedings of the 10th Widening NLP Workshop
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Chen Zhang, Emily Allaway, Meryem M’hamdi, Peerat Limkonchotiwat, Richard He Bai, Santosh T.Y.S.S., Surendrabikram Thapa, Yinqiao Li
Venues:
WiNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
105–120
Language:
URL:
https://aclanthology.org/2026.winlp-main.11/
DOI:
Bibkey:
Cite (ACL):
Soufia Bahmani and Keith Vertanen. 2026. Improved Personalized Typing Predictions via Long Left Context. In Proceedings of the 10th Widening NLP Workshop, pages 105–120, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Improved Personalized Typing Predictions via Long Left Context (Bahmani & Vertanen, WiNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.winlp-main.11.pdf