Avinash Goutham Aluguvelly

Author directory

2026

We audit whether outperforming random activation directions establishes improved agent per- formance. Using Llama-3.1-8B in a molecular identification task, we compute every legal action’s exact probability of eventual success un- der optimal continuation. An entropy-weighted steering direction outperforms all 39 layer- and norm-matched random controls on 42 held-out molecular pools in a frozen comparison. Posthoc audits reveal the limits of this result. An untuned policy that chooses uniformly among legal actions also outperforms every random control, with no detectable difference from the steered policy; uncertainty intervals still admit modest effects in either direction. The steering direction ranks 10th among 36 directions when controls are matched to its construction, and its original-bank rank falls to 3/40 when pools sharing chemical scaffolds are grouped and weighted equally. Executed episodes show no detectable improvement over the unsteered baseline, with both policies performing near random under constrained decoding. These findings concern one model and task; they do not establish that steering cannot improve a competent agent. They demonstrate that su- periority to a random-direction bank does not by itself establish policy improvement. We propose five reporting controls covering trivial baselines, direction construction, inferential units, state selection and horizon, and executed behavior.
Search
Co-authors
    Venues
    Fix author