<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lee-etal-2026-fol">
    <titleInfo>
        <title>FOL-Traces: Verified First-Order Logic Reasoning Traces at Scale</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Isabelle</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sarah</namePart>
        <namePart type="family">Liaw</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dani</namePart>
        <namePart type="family">Yogatama</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-03</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EACL 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Vera</namePart>
            <namePart type="family">Demberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kentaro</namePart>
            <namePart type="family">Inui</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lluís</namePart>
            <namePart type="family">Marquez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Rabat, Morocco</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-386-9</identifier>
    </relatedItem>
    <abstract>Reasoning in language models is difficult to evaluate: natural-language traces are unverifiable, symbolic datasets are too small, and most benchmarks conflate heuristics with inference. We present FOL-Traces, the first large-scale dataset of programmatically verified reasoning traces, enabling rigorous evaluation of structured logical inference. We also propose two challenging and comprehensive diagnostic tasks—masked operation prediction and step completion—that directly probe syntactic awareness and process fidelity. FOL-Traces serves as a scalable testbed for rigorously studying how models perform structured logical inference. Systematic experiments with 5 reasoning LLMs show that the dataset remains challenging: models only reach around 45.7% accuracy on masked operation prediction and around 27% on two-step completion.</abstract>
    <identifier type="citekey">lee-etal-2026-fol</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-eacl.115/</url>
    </location>
    <part>
        <date>2026-03</date>
        <extent unit="page">
            <start>2181</start>
            <end>2203</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T FOL-Traces: Verified First-Order Logic Reasoning Traces at Scale
%A Lee, Isabelle
%A Liaw, Sarah
%A Yogatama, Dani
%Y Demberg, Vera
%Y Inui, Kentaro
%Y Marquez, Lluís
%S Findings of the Association for Computational Linguistics: EACL 2026
%D 2026
%8 March
%I Association for Computational Linguistics
%C Rabat, Morocco
%@ 979-8-89176-386-9
%F lee-etal-2026-fol
%X Reasoning in language models is difficult to evaluate: natural-language traces are unverifiable, symbolic datasets are too small, and most benchmarks conflate heuristics with inference. We present FOL-Traces, the first large-scale dataset of programmatically verified reasoning traces, enabling rigorous evaluation of structured logical inference. We also propose two challenging and comprehensive diagnostic tasks—masked operation prediction and step completion—that directly probe syntactic awareness and process fidelity. FOL-Traces serves as a scalable testbed for rigorously studying how models perform structured logical inference. Systematic experiments with 5 reasoning LLMs show that the dataset remains challenging: models only reach around 45.7% accuracy on masked operation prediction and around 27% on two-step completion.
%U https://aclanthology.org/2026.findings-eacl.115/
%P 2181-2203

Markdown (Informal)

[FOL-Traces: Verified First-Order Logic Reasoning Traces at Scale](https://aclanthology.org/2026.findings-eacl.115/) (Lee et al., Findings 2026)

• FOL-Traces: Verified First-Order Logic Reasoning Traces at Scale (Lee et al., Findings 2026)

ACL

• Isabelle Lee, Sarah Liaw, and Dani Yogatama. 2026. FOL-Traces: Verified First-Order Logic Reasoning Traces at Scale. In Findings of the Association for Computational Linguistics: EACL 2026, pages 2181–2203, Rabat, Morocco. Association for Computational Linguistics.