Artem Cherepanov
Author directory2026
What Should Occupy the Context Budget? Latent Compression versus Budget-Matched Truncation in Document QA
Artem Cherepanov | Albina Burlova
The Proceedings of Workshop on Document Intelligence and Understanding
Artem Cherepanov | Albina Burlova
The Proceedings of Workshop on Document Intelligence and Understanding
Latent context compression replaces a document with a short sequence of learned vectors, but whether this beats simply keeping raw tokens under the same decoder-context budget is rarely tested. We run that test on held-out document QA, matching budgets per example against raw-token truncation and selector-specific readers trained on first-N, LLMLingua-2, and question-conditioned BM25 contexts. Our minimal compressor consists of a text encoder, learned pooling, and direct projection. It beats every tested common-cap retention baseline at nominal pooling factors cr=8 and cr=16, reaching answer perplexity 3.464 versus 5.349 for the strongest baseline at cr=8. The advantage persists in first-sentence free generation, and a manual check of answers is directionally consistent. Adding a post-pooling transformer hurts with twelve layers and with one layer. The twelve-layer result holds in the discovery run, two replications, and a second decoder family. Removing the post-pooling transformer also makes the compressor smaller and faster. Learned pooling followed by direct projection provides the strongest quality and efficiency among the tested compressor designs.
2025
Steering LLM Reasoning Through Bias-Only Adaptation
Viacheslav Sinii | Alexey Gorbatovski | Artem Cherepanov | Boris Shaposhnikov | Nikita Balagansky | Daniil Gavrilov
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Viacheslav Sinii | Alexey Gorbatovski | Artem Cherepanov | Boris Shaposhnikov | Nikita Balagansky | Daniil Gavrilov
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
We show that training a single d-dimensional steering vector per layer with reinforcement learning, while freezing all base weights, matches the accuracy of fully RL-tuned reasoning models on mathematical-reasoning tasks.On an 8 billion-parameter model this adds only ≈ 0.0016% additional parameters and reproduces performance across a range of base models and mathematical-reasoning benchmarks.These results tighten the upper bound on the parameter budget required for high-level chain-of-thought reasoning, indicating that millions of adapter weights are unnecessary.The minimal trainable footprint reduces optimizer memory and inter-GPU communication, lowering the overall cost of fine-tuning.Moreover, a logit-lens analysis shows that the learned vectors amplify coherent token directions, providing clearer insight into the model’s internal computations.