Artem Cherepanov

Author directory

2026

Latent context compression replaces a document with a short sequence of learned vectors, but whether this beats simply keeping raw tokens under the same decoder-context budget is rarely tested. We run that test on held-out document QA, matching budgets per example against raw-token truncation and selector-specific readers trained on first-N, LLMLingua-2, and question-conditioned BM25 contexts. Our minimal compressor consists of a text encoder, learned pooling, and direct projection. It beats every tested common-cap retention baseline at nominal pooling factors cr=8 and cr=16, reaching answer perplexity 3.464 versus 5.349 for the strongest baseline at cr=8. The advantage persists in first-sentence free generation, and a manual check of answers is directionally consistent. Adding a post-pooling transformer hurts with twelve layers and with one layer. The twelve-layer result holds in the discovery run, two replications, and a second decoder family. Removing the post-pooling transformer also makes the compressor smaller and faster. Learned pooling followed by direct projection provides the strongest quality and efficiency among the tested compressor designs.

2025

We show that training a single d-dimensional steering vector per layer with reinforcement learning, while freezing all base weights, matches the accuracy of fully RL-tuned reasoning models on mathematical-reasoning tasks.On an 8 billion-parameter model this adds only ≈ 0.0016% additional parameters and reproduces performance across a range of base models and mathematical-reasoning benchmarks.These results tighten the upper bound on the parameter budget required for high-level chain-of-thought reasoning, indicating that millions of adapter weights are unnecessary.The minimal trainable footprint reduces optimizer memory and inter-GPU communication, lowering the overall cost of fine-tuning.Moreover, a logit-lens analysis shows that the learned vectors amplify coherent token directions, providing clearer insight into the model’s internal computations.