<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lee-kim-2023-recursion">
    <titleInfo>
        <title>Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Soochan</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Gunhee</namePart>
        <namePart type="family">Kim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Generating intermediate steps, or Chain of Thought (CoT), is an effective way to significantly improve language models’ (LM) multi-step reasoning capability. However, the CoT lengths can grow rapidly with the problem complexity, easily exceeding the maximum context size. Instead of increasing the context limit, which has already been heavily investigated, we explore an orthogonal direction: making LMs divide a problem into multiple contexts. We propose a new inference framework, called Recursion of Thought (RoT), which introduces several special tokens that the models can output to trigger context-related operations. Extensive experiments with multiple architectures including GPT-3 show that RoT dramatically improves LMs’ inference capability to solve problems, whose solution consists of hundreds of thousands of tokens.</abstract>
    <identifier type="citekey">lee-kim-2023-recursion</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-acl.40</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-acl.40</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>623</start>
            <end>658</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models
%A Lee, Soochan
%A Kim, Gunhee
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Findings of the Association for Computational Linguistics: ACL 2023
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F lee-kim-2023-recursion
%X Generating intermediate steps, or Chain of Thought (CoT), is an effective way to significantly improve language models’ (LM) multi-step reasoning capability. However, the CoT lengths can grow rapidly with the problem complexity, easily exceeding the maximum context size. Instead of increasing the context limit, which has already been heavily investigated, we explore an orthogonal direction: making LMs divide a problem into multiple contexts. We propose a new inference framework, called Recursion of Thought (RoT), which introduces several special tokens that the models can output to trigger context-related operations. Extensive experiments with multiple architectures including GPT-3 show that RoT dramatically improves LMs’ inference capability to solve problems, whose solution consists of hundreds of thousands of tokens.
%R 10.18653/v1/2023.findings-acl.40
%U https://aclanthology.org/2023.findings-acl.40
%U https://doi.org/10.18653/v1/2023.findings-acl.40
%P 623-658

Markdown (Informal)

[Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models](https://aclanthology.org/2023.findings-acl.40) (Lee & Kim, Findings 2023)

• Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models (Lee & Kim, Findings 2023)

ACL

• Soochan Lee and Gunhee Kim. 2023. Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models. In Findings of the Association for Computational Linguistics: ACL 2023, pages 623–658, Toronto, Canada. Association for Computational Linguistics.