<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="jiao-etal-2026-policy">
    <titleInfo>
        <title>Policy of Thoughts: Scaling LLM Reasoning via Test-time Policy Evolution</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Zhengbo</namePart>
        <namePart type="family">Jiao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hongyu</namePart>
        <namePart type="family">Xian</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Qinglong</namePart>
        <namePart type="family">Wang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yunpu</namePart>
        <namePart type="family">Ma</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Zhebo</namePart>
        <namePart type="family">Wang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dezhang</namePart>
        <namePart type="family">Kong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Meng</namePart>
        <namePart type="family">Han</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Large language models (LLMs) struggle with complex, long-horizon reasoning due to instability from the frozen-policy assumption. Current test-time scaling methods treat execution feedback merely as an external signal for filtering or rewriting trajectories, without internalizing it to improve reasoning strategies. Inspired by Popper’s “conjectures and refutations,” we posit that effective reasoning requires real-time policy adaptation through learning from failures. We introduce Policy of Thoughts (PoT), a framework that recasts reasoning as a within-instance online optimization process. PoT effectively generates diverse candidates via targeted exploration, then applies Group Relative Policy Optimization (GRPO) to update a transient LoRA adapter using execution feedback. This closed-loop design enables dynamic, instance-specific refinement of reasoning priors. Experiments demonstrate PoT’s substantial gains across multiple benchmarks. On LiveCodeBench, a 4B model achieves 49.71% accuracy, surpassing 12 test-time scaling baselines and commercial models (e.g., GPT-4o, DeepSeek-V3) despite being over 50 smaller.</abstract>
    <identifier type="citekey">jiao-etal-2026-policy</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.1908/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>43472</start>
            <end>43490</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Policy of Thoughts: Scaling LLM Reasoning via Test-time Policy Evolution
%A Jiao, Zhengbo
%A Xian, Hongyu
%A Wang, Qinglong
%A Ma, Yunpu
%A Wang, Zhebo
%A Kong, Dezhang
%A Han, Meng
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F jiao-etal-2026-policy
%X Large language models (LLMs) struggle with complex, long-horizon reasoning due to instability from the frozen-policy assumption. Current test-time scaling methods treat execution feedback merely as an external signal for filtering or rewriting trajectories, without internalizing it to improve reasoning strategies. Inspired by Popper’s “conjectures and refutations,” we posit that effective reasoning requires real-time policy adaptation through learning from failures. We introduce Policy of Thoughts (PoT), a framework that recasts reasoning as a within-instance online optimization process. PoT effectively generates diverse candidates via targeted exploration, then applies Group Relative Policy Optimization (GRPO) to update a transient LoRA adapter using execution feedback. This closed-loop design enables dynamic, instance-specific refinement of reasoning priors. Experiments demonstrate PoT’s substantial gains across multiple benchmarks. On LiveCodeBench, a 4B model achieves 49.71% accuracy, surpassing 12 test-time scaling baselines and commercial models (e.g., GPT-4o, DeepSeek-V3) despite being over 50 smaller.
%U https://aclanthology.org/2026.emnlp-main.1908/
%P 43472-43490

Markdown (Informal)

[Policy of Thoughts: Scaling LLM Reasoning via Test-time Policy Evolution](https://aclanthology.org/2026.emnlp-main.1908/) (Jiao et al., EMNLP 2026)

• Policy of Thoughts: Scaling LLM Reasoning via Test-time Policy Evolution (Jiao et al., EMNLP 2026)

ACL

• Zhengbo Jiao, Hongyu Xian, Qinglong Wang, Yunpu Ma, Zhebo Wang, Dezhang Kong, and Meng Han. 2026. Policy of Thoughts: Scaling LLM Reasoning via Test-time Policy Evolution. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 43472–43490, Budapest, Hungary. Association for Computational Linguistics.