<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="itzhak-etal-2024-instructed">
    <titleInfo>
        <title>Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Itay</namePart>
        <namePart type="family">Itzhak</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Gabriel</namePart>
        <namePart type="family">Stanovsky</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nir</namePart>
        <namePart type="family">Rosenfeld</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yonatan</namePart>
        <namePart type="family">Belinkov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>Recent studies show that instruction tuning (IT) and reinforcement learning from human feedback (RLHF) improve the abilities of large language models (LMs) dramatically. While these tuning methods can help align models with human objectives and generate high-quality text, not much is known about their potential adverse effects. In this work, we investigate the effect of IT and RLHF on decision making and reasoning in LMs, focusing on three cognitive biases—the decoy effect, the certainty effect, and the belief bias—all of which are known to influence human decision-making and reasoning. Our findings highlight the presence of these biases in various models from the GPT-3, Mistral, and T5 families. Notably, we find a stronger presence of biases in models that have undergone instruction tuning, such as Flan-T5, Mistral-Instruct, GPT3.5, and GPT4. Our work constitutes a step toward comprehending cognitive biases in instruction-tuned LMs, which is crucial for the development of more reliable and unbiased language models.1</abstract>
    <identifier type="citekey">itzhak-etal-2024-instructed</identifier>
    <identifier type="doi">10.1162/tacl_a_00673</identifier>
    <location>
        <url>https://aclanthology.org/2024.tacl-1.43</url>
    </location>
    <part>
        <date>2024</date>
        <detail type="volume"><number>12</number></detail>
        <extent unit="page">
            <start>771</start>
            <end>785</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias
%A Itzhak, Itay
%A Stanovsky, Gabriel
%A Rosenfeld, Nir
%A Belinkov, Yonatan
%J Transactions of the Association for Computational Linguistics
%D 2024
%V 12
%I MIT Press
%C Cambridge, MA
%F itzhak-etal-2024-instructed
%X Recent studies show that instruction tuning (IT) and reinforcement learning from human feedback (RLHF) improve the abilities of large language models (LMs) dramatically. While these tuning methods can help align models with human objectives and generate high-quality text, not much is known about their potential adverse effects. In this work, we investigate the effect of IT and RLHF on decision making and reasoning in LMs, focusing on three cognitive biases—the decoy effect, the certainty effect, and the belief bias—all of which are known to influence human decision-making and reasoning. Our findings highlight the presence of these biases in various models from the GPT-3, Mistral, and T5 families. Notably, we find a stronger presence of biases in models that have undergone instruction tuning, such as Flan-T5, Mistral-Instruct, GPT3.5, and GPT4. Our work constitutes a step toward comprehending cognitive biases in instruction-tuned LMs, which is crucial for the development of more reliable and unbiased language models.1
%R 10.1162/tacl_a_00673
%U https://aclanthology.org/2024.tacl-1.43
%U https://doi.org/10.1162/tacl_a_00673
%P 771-785

Markdown (Informal)

[Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias](https://aclanthology.org/2024.tacl-1.43) (Itzhak et al., TACL 2024)

• Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias (Itzhak et al., TACL 2024)

ACL

• Itay Itzhak, Gabriel Stanovsky, Nir Rosenfeld, and Yonatan Belinkov. 2024. Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias. Transactions of the Association for Computational Linguistics, 12:771–785.