<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="haque-etal-2025-towards">
    <titleInfo>
        <title>Towards Effectively Leveraging Execution Traces for Program Repair with Code LLMs</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Mirazul</namePart>
        <namePart type="family">Haque</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Petr</namePart>
        <namePart type="family">Babkin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Farima</namePart>
        <namePart type="family">Farmahinifarahani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Manuela</namePart>
        <namePart type="family">Veloso</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Weijia</namePart>
            <namePart type="family">Shi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Wenhao</namePart>
            <namePart type="family">Yu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Akari</namePart>
            <namePart type="family">Asai</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Meng</namePart>
            <namePart type="family">Jiang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Greg</namePart>
            <namePart type="family">Durrett</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hannaneh</namePart>
            <namePart type="family">Hajishirzi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Luke</namePart>
            <namePart type="family">Zettlemoyer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Albuquerque, New Mexico, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-229-9</identifier>
    </relatedItem>
    <abstract>Large Language Models (LLMs) show promising performance on various programming tasks, including Automatic Program Repair (APR).However, most approaches to LLM-based APR are limited to the static analysis of the programs, while disregarding their runtime behavior.Inspired by knowledge-augmented NLP, in this work, we aim to remedy this potential blind spot by augmenting standard APR prompts with program execution traces.We evaluate our approach using the GPT family of models on three popular APR datasets. Our findings suggest that simply incorporating execution traces into the prompt provides a limited performance improvement over trace-free baselines, in only 2 out of 6 tested dataset/model configurations. We further find that the effectiveness of execution traces for APR diminishes as their complexity increases. We explore several strategies for leveraging traces in promptsand demonstrate that LLM-optimized prompts help outperform trace-free prompts more consistently.Additionally, we show trace-based prompting to be superior to finetuning a smaller LLM on a small-scale dataset; and conduct probing studies reinforcing the notion that execution traces can complement the reasoning abilities of the LLMs.</abstract>
    <identifier type="citekey">haque-etal-2025-towards</identifier>
    <identifier type="doi">10.18653/v1/2025.knowledgenlp-1.17</identifier>
    <location>
        <url>https://aclanthology.org/2025.knowledgenlp-1.17/</url>
    </location>
    <part>
        <date>2025-05</date>
        <extent unit="page">
            <start>160</start>
            <end>179</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Towards Effectively Leveraging Execution Traces for Program Repair with Code LLMs
%A Haque, Mirazul
%A Babkin, Petr
%A Farmahinifarahani, Farima
%A Veloso, Manuela
%Y Shi, Weijia
%Y Yu, Wenhao
%Y Asai, Akari
%Y Jiang, Meng
%Y Durrett, Greg
%Y Hajishirzi, Hannaneh
%Y Zettlemoyer, Luke
%S Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing
%D 2025
%8 May
%I Association for Computational Linguistics
%C Albuquerque, New Mexico, USA
%@ 979-8-89176-229-9
%F haque-etal-2025-towards
%X Large Language Models (LLMs) show promising performance on various programming tasks, including Automatic Program Repair (APR).However, most approaches to LLM-based APR are limited to the static analysis of the programs, while disregarding their runtime behavior.Inspired by knowledge-augmented NLP, in this work, we aim to remedy this potential blind spot by augmenting standard APR prompts with program execution traces.We evaluate our approach using the GPT family of models on three popular APR datasets. Our findings suggest that simply incorporating execution traces into the prompt provides a limited performance improvement over trace-free baselines, in only 2 out of 6 tested dataset/model configurations. We further find that the effectiveness of execution traces for APR diminishes as their complexity increases. We explore several strategies for leveraging traces in promptsand demonstrate that LLM-optimized prompts help outperform trace-free prompts more consistently.Additionally, we show trace-based prompting to be superior to finetuning a smaller LLM on a small-scale dataset; and conduct probing studies reinforcing the notion that execution traces can complement the reasoning abilities of the LLMs.
%R 10.18653/v1/2025.knowledgenlp-1.17
%U https://aclanthology.org/2025.knowledgenlp-1.17/
%U https://doi.org/10.18653/v1/2025.knowledgenlp-1.17
%P 160-179

Markdown (Informal)

[Towards Effectively Leveraging Execution Traces for Program Repair with Code LLMs](https://aclanthology.org/2025.knowledgenlp-1.17/) (Haque et al., KnowledgeNLP 2025)

• Towards Effectively Leveraging Execution Traces for Program Repair with Code LLMs (Haque et al., KnowledgeNLP 2025)

ACL

• Mirazul Haque, Petr Babkin, Farima Farmahinifarahani, and Manuela Veloso. 2025. Towards Effectively Leveraging Execution Traces for Program Repair with Code LLMs. In Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing, pages 160–179, Albuquerque, New Mexico, USA. Association for Computational Linguistics.