<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="wan-etal-2026-retrieving">
    <titleInfo>
        <title>Retrieving Facts or Reinforcing Beliefs? Towards Understanding Confirmation Bias in RAG Systems for Scientific Claim Verification</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Qile</namePart>
        <namePart type="family">Wan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Davide</namePart>
        <namePart type="family">Venditti</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Fabio</namePart>
        <namePart type="given">Massimo</namePart>
        <namePart type="family">Zanzotto</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Iryna</namePart>
        <namePart type="family">Gurevych</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lizhen</namePart>
        <namePart type="family">Qu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>The rapid growth of scientific literature has made automated verification of scientific claims increasingly important, with Retrieval-Augmented Generation (RAG) systems emerging as a promising solution. However, their multi-stage architecture raises concerns about confirmation bias, where systems preferentially retrieve and generate evidence that supports a given claim. In this paper, we present CoBiT (Confirmation Bias Test), the first benchmark for systematically evaluating confirmation bias in RAG systems, consisting of 707 minimally perturbed claim–counterclaim pairs. We propose a unified framework with stage-specific metrics to quantify bias in both re-ranking and generation, including Ranking Bias Magnitude (RBM), Confirmation Bias Rate (CBR), and consistency measures. Our experiments on OpenScholar,ScholarQA, and LLM baselines reveal that confirmation bias arises at different stages of the RAG pipeline and can be amplified by re-ranking and generation. In particular, SciRAG achieves the lowest generation bias (CBR = 0.13), it exhibits higher bias in re-ranking, and OpenScholar shows the greatest inconsistency between claims and counterclaims. These findings highlight the need for fine-grained evaluation of epistemic risks in RAG systems and provide practical tools for diagnosing confirmation bias.\par Code and Data https://github.com/Presence76/Confirmation_bias_evaluation</abstract>
    <identifier type="citekey">wan-etal-2026-retrieving</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.44/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>958</start>
            <end>974</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Retrieving Facts or Reinforcing Beliefs? Towards Understanding Confirmation Bias in RAG Systems for Scientific Claim Verification
%A Wan, Qile
%A Venditti, Davide
%A Zanzotto, Fabio Massimo
%A Gurevych, Iryna
%A Qu, Lizhen
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F wan-etal-2026-retrieving
%X The rapid growth of scientific literature has made automated verification of scientific claims increasingly important, with Retrieval-Augmented Generation (RAG) systems emerging as a promising solution. However, their multi-stage architecture raises concerns about confirmation bias, where systems preferentially retrieve and generate evidence that supports a given claim. In this paper, we present CoBiT (Confirmation Bias Test), the first benchmark for systematically evaluating confirmation bias in RAG systems, consisting of 707 minimally perturbed claim–counterclaim pairs. We propose a unified framework with stage-specific metrics to quantify bias in both re-ranking and generation, including Ranking Bias Magnitude (RBM), Confirmation Bias Rate (CBR), and consistency measures. Our experiments on OpenScholar,ScholarQA, and LLM baselines reveal that confirmation bias arises at different stages of the RAG pipeline and can be amplified by re-ranking and generation. In particular, SciRAG achieves the lowest generation bias (CBR = 0.13), it exhibits higher bias in re-ranking, and OpenScholar shows the greatest inconsistency between claims and counterclaims. These findings highlight the need for fine-grained evaluation of epistemic risks in RAG systems and provide practical tools for diagnosing confirmation bias.\par Code and Data https://github.com/Presence76/Confirmation_bias_evaluation
%U https://aclanthology.org/2026.emnlp-main.44/
%P 958-974

Markdown (Informal)

[Retrieving Facts or Reinforcing Beliefs? Towards Understanding Confirmation Bias in RAG Systems for Scientific Claim Verification](https://aclanthology.org/2026.emnlp-main.44/) (Wan et al., EMNLP 2026)

• Retrieving Facts or Reinforcing Beliefs? Towards Understanding Confirmation Bias in RAG Systems for Scientific Claim Verification (Wan et al., EMNLP 2026)

ACL

• Qile Wan, Davide Venditti, Fabio Massimo Zanzotto, Iryna Gurevych, and Lizhen Qu. 2026. Retrieving Facts or Reinforcing Beliefs? Towards Understanding Confirmation Bias in RAG Systems for Scientific Claim Verification. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 958–974, Budapest, Hungary. Association for Computational Linguistics.