Frédéric Berdoz

Author directory

2026

One-shot block drafters for speculative decoding generate the full draft in a single forward pass, achieving strong throughput by eliminating sequential token generation. However, they predict each draft token conditioned only on the prefix context, with no dependence on previously drafted tokens. This non-autoregressive conditioning causes the drafter’s distribution to diverge from the verifier’s true autoregressive distribution as draft depth grows. This problem becomes more severe in tree-based drafting, where distinct branches are forced to share the same marginal distribution for subsequent tokens. We propose TreeFlash, which addresses this by incorporating an MLP layer conditioned on the drafter’s hidden state and the previous token to approximate an autoregressive distribution. TreeFlash retains the 𝒪(1) decoding time complexity of one-shot drafters by employing a two-stage approximation mechanism. TreeFlash achieves state-of-the-art performance across a variety of tasks and models, improving over marginal tree drafting by 12% higher block efficiency and 9% higher speedup at a drafting budget of 64.
Modeling and predicting political opinions remains a challenging task for AI systems. We introduce PoliSpace, a framework that embeds individuals and political questions into a shared latent space, where vector alignment directly reflects agreement. Individuals are represented as learnable vectors, and questions are encoded via a contrastively fine-tuned language model with individual-specific affine transformations to capture subjective interpretation. This design yields an embedding space that is robust to question rephrasing and sensitive to negation. Evaluated on real-world voting data from Swiss national elections, PoliSpace achieves an 89.81% F1 score on seen opinion prediction for rephrased questions, outperforming prior embedding-based approaches (66.17% F1). We further introduce retrieval and confidence-aware thresholding mechanisms that reach up to 100% F1 at reduced coverage, demonstrating how the learned embedding space can be leveraged to improve prediction reliability. Ablation studies confirm the importance of both individual-specific interpretation modeling and negation-aware encoder fine-tuning. The learned individual embedding space also exhibits clear clustering by political party affiliation, despite no party-label supervision during training, capturing the underlying ideological structure of the Swiss political landscape.