<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="shapira-etal-2025-human">
    <titleInfo>
        <title>Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Eilam</namePart>
        <namePart type="family">Shapira</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Omer</namePart>
        <namePart type="family">Madmon</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Reut</namePart>
        <namePart type="family">Apel</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Moshe</namePart>
        <namePart type="family">Tennenholtz</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Roi</namePart>
        <namePart type="family">Reichart</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>Recent advances in Large Language Models (LLMs) have spurred interest in designing LLM-based agents for tasks that involve interaction with human and artificial agents. This paper addresses a key aspect in the design of such agents: predicting human decisions in off-policy evaluation (OPE). We focus on language-based persuasion games, where an expert aims to influence the decision-maker through verbal messages. In our OPE framework, the prediction model is trained on human interaction data collected from encounters with one set of expert agents, and its performance is evaluated on interactions with a different set of experts. Using a dedicated application, we collected a dataset of 87K decisions from humans playing a repeated decision-making game with artificial agents. To enhance off-policy performance, we propose a simulation technique involving interactions across the entire agent space and simulated decision-makers. Our learning strategy yields significant OPE gains, e.g., improving prediction accuracy in the top 15% challenging cases by 7.1%.1</abstract>
    <identifier type="citekey">shapira-etal-2025-human</identifier>
    <identifier type="doi">10.1162/tacl.a.16</identifier>
    <location>
        <url>https://aclanthology.org/2025.tacl-1.46/</url>
    </location>
    <part>
        <date>2025</date>
        <detail type="volume"><number>13</number></detail>
        <extent unit="page">
            <start>980</start>
            <end>1006</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluation
%A Shapira, Eilam
%A Madmon, Omer
%A Apel, Reut
%A Tennenholtz, Moshe
%A Reichart, Roi
%J Transactions of the Association for Computational Linguistics
%D 2025
%V 13
%I MIT Press
%C Cambridge, MA
%F shapira-etal-2025-human
%X Recent advances in Large Language Models (LLMs) have spurred interest in designing LLM-based agents for tasks that involve interaction with human and artificial agents. This paper addresses a key aspect in the design of such agents: predicting human decisions in off-policy evaluation (OPE). We focus on language-based persuasion games, where an expert aims to influence the decision-maker through verbal messages. In our OPE framework, the prediction model is trained on human interaction data collected from encounters with one set of expert agents, and its performance is evaluated on interactions with a different set of experts. Using a dedicated application, we collected a dataset of 87K decisions from humans playing a repeated decision-making game with artificial agents. To enhance off-policy performance, we propose a simulation technique involving interactions across the entire agent space and simulated decision-makers. Our learning strategy yields significant OPE gains, e.g., improving prediction accuracy in the top 15% challenging cases by 7.1%.1
%R 10.1162/tacl.a.16
%U https://aclanthology.org/2025.tacl-1.46/
%U https://doi.org/10.1162/tacl.a.16
%P 980-1006

Markdown (Informal)

[Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluation](https://aclanthology.org/2025.tacl-1.46/) (Shapira et al., TACL 2025)

• Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluation (Shapira et al., TACL 2025)

ACL

• Eilam Shapira, Omer Madmon, Reut Apel, Moshe Tennenholtz, and Roi Reichart. 2025. Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluation. Transactions of the Association for Computational Linguistics, 13:980–1006.