<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="dutta-etal-2024-applying">
    <titleInfo>
        <title>Applying RLAIF for Code Generation with API-usage in Lightweight LLMs</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sujan</namePart>
        <namePart type="family">Dutta</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sayantan</namePart>
        <namePart type="family">Mahinder</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Raviteja</namePart>
        <namePart type="family">Anantha</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Bortik</namePart>
        <namePart type="family">Bandyopadhyay</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2nd Workshop on Natural Language Reasoning and Structured Explanations (@ACL 2024)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Bhavana</namePart>
            <namePart type="family">Dalvi Mishra</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Greg</namePart>
            <namePart type="family">Durrett</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Peter</namePart>
            <namePart type="family">Jansen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ben</namePart>
            <namePart type="family">Lipkin</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Danilo</namePart>
            <namePart type="family">Neves Ribeiro</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lionel</namePart>
            <namePart type="family">Wong</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Xi</namePart>
            <namePart type="family">Ye</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Wenting</namePart>
            <namePart type="family">Zhao</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Reinforcement Learning from AI Feedback (RLAIF) has demonstrated significant potential across various domains, including mitigating harm in LLM outputs, enhancing text summarization, and mathematical reasoning. This paper introduces an RLAIF framework for improving the code generation abilities of lightweight (\textless1B parameters) LLMs. We specifically focus on code generation tasks that require writing appropriate API calls, which is challenging due to the well-known issue of hallucination in LLMs. Our framework extracts AI feedback from a larger LLM (e.g., GPT-3.5) through a specialized prompting strategy and uses this data to train a reward model towards better alignment from smaller LLMs. We run our experiments on the Gorilla dataset and meticulously assess the quality of the model-generated code across various metrics, including AST, ROUGE, and Code-BLEU, and develop a pipeline to compute its executability rate accurately. Our approach significantly enhances the fine-tuned LLM baseline’s performance, achieving a 4.5% improvement in executability rate. Notably, a smaller LLM model (780M parameters) trained with RLAIF surpasses a much larger fine-tuned baseline with 7B parameters, achieving a 1.0% higher code executability rate.</abstract>
    <identifier type="citekey">dutta-etal-2024-applying</identifier>
    <location>
        <url>https://aclanthology.org/2024.nlrse-1.4</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>39</start>
            <end>45</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Applying RLAIF for Code Generation with API-usage in Lightweight LLMs
%A Dutta, Sujan
%A Mahinder, Sayantan
%A Anantha, Raviteja
%A Bandyopadhyay, Bortik
%Y Dalvi Mishra, Bhavana
%Y Durrett, Greg
%Y Jansen, Peter
%Y Lipkin, Ben
%Y Neves Ribeiro, Danilo
%Y Wong, Lionel
%Y Ye, Xi
%Y Zhao, Wenting
%S Proceedings of the 2nd Workshop on Natural Language Reasoning and Structured Explanations (@ACL 2024)
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F dutta-etal-2024-applying
%X Reinforcement Learning from AI Feedback (RLAIF) has demonstrated significant potential across various domains, including mitigating harm in LLM outputs, enhancing text summarization, and mathematical reasoning. This paper introduces an RLAIF framework for improving the code generation abilities of lightweight (\textless1B parameters) LLMs. We specifically focus on code generation tasks that require writing appropriate API calls, which is challenging due to the well-known issue of hallucination in LLMs. Our framework extracts AI feedback from a larger LLM (e.g., GPT-3.5) through a specialized prompting strategy and uses this data to train a reward model towards better alignment from smaller LLMs. We run our experiments on the Gorilla dataset and meticulously assess the quality of the model-generated code across various metrics, including AST, ROUGE, and Code-BLEU, and develop a pipeline to compute its executability rate accurately. Our approach significantly enhances the fine-tuned LLM baseline’s performance, achieving a 4.5% improvement in executability rate. Notably, a smaller LLM model (780M parameters) trained with RLAIF surpasses a much larger fine-tuned baseline with 7B parameters, achieving a 1.0% higher code executability rate.
%U https://aclanthology.org/2024.nlrse-1.4
%P 39-45

Markdown (Informal)

[Applying RLAIF for Code Generation with API-usage in Lightweight LLMs](https://aclanthology.org/2024.nlrse-1.4) (Dutta et al., NLRSE-WS 2024)

• Applying RLAIF for Code Generation with API-usage in Lightweight LLMs (Dutta et al., NLRSE-WS 2024)

ACL

• Sujan Dutta, Sayantan Mahinder, Raviteja Anantha, and Bortik Bandyopadhyay. 2024. Applying RLAIF for Code Generation with API-usage in Lightweight LLMs. In Proceedings of the 2nd Workshop on Natural Language Reasoning and Structured Explanations (@ACL 2024), pages 39–45, Bangkok, Thailand. Association for Computational Linguistics.