We present AgentLens, a trajectory-level evaluation framework for interactive code agents, built from production-informed tasks and integrated into continuous regression testing. Most code-agent benchmarks reduce a run to a single bit—did the task pass?—but the people who actually use these agents experience the entire trajectory: how the agent follows instructions, uses its tools, verifies its own work, recovers from mistakes, and talks to them along the way. AgentLens evaluates that whole trajectory, pairing formal verification with LLM-written trajectory reviews and side-by-side comparisons, so that each run yields a readable explanation of the final score. Apart from ranking models, AgentLens can be used to diagnose model behavior, compare successive versions of agents, and catch product regressions in an evaluation pipeline. We release AgentLens at https://github.com/agent-lens/agent-lens-bench.

We propose Project Context for Code Summarization with LLMs (ProConSuL), a new framework to provide a large language model (LLM) with precise information about the code structure from program analysis methods such as a compiler or IDE language services and use task decomposition derived from the code structure. ProConSuL builds a call graph to provide the context from callees and uses a two-phase training method (SFT + preference alignment) to train the model to use the project context. We also provide a new evaluation benchmark for C/C++ functions and a set of proxy metrics. Experimental results demonstrate that ProConSuL allows to significantly improve code summaries and reduce the number of hallucinations compared to the base model (CodeLlama-7B-instruct). We make our code and dataset available at https://github.com/TypingCat13/ProConSuL.