Maksim Parshin
Author directory2026
AgentLens: Trajectory Reviews for Coding Agent Evaluation
Andrey Podivilov | Vadim Lomshakov | Sergey Savin | Matvey Startsev | Roman Pozharskii | Maksim Parshin | Sergey Nikolenko
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Andrey Podivilov | Vadim Lomshakov | Sergey Savin | Matvey Startsev | Roman Pozharskii | Maksim Parshin | Sergey Nikolenko
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
We present AgentLens, a trajectory-level evaluation framework for interactive code agents, built from production-informed tasks and integrated into continuous regression testing. Most code-agent benchmarks reduce a run to a single bit—did the task pass?—but the people who actually use these agents experience the entire trajectory: how the agent follows instructions, uses its tools, verifies its own work, recovers from mistakes, and talks to them along the way. AgentLens evaluates that whole trajectory, pairing formal verification with LLM-written trajectory reviews and side-by-side comparisons, so that each run yields a readable explanation of the final score. Apart from ranking models, AgentLens can be used to diagnose model behavior, compare successive versions of agents, and catch product regressions in an evaluation pipeline. We release AgentLens at https://github.com/agent-lens/agent-lens-bench.