<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="xie-etal-2026-preliminary">
    <titleInfo>
        <title>Preliminary Study on LLM-as-a-Judge Evaluation of Causal Mechanism Explanations in Financial News and its Bias Analysis</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yucheng</namePart>
        <namePart type="family">Xie</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yurina</namePart>
        <namePart type="family">Kosai</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rikuto</namePart>
        <namePart type="family">Tsuchida</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Takehito</namePart>
        <namePart type="family">Utsuro</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Jimin</namePart>
            <namePart type="family">Huang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yongjae</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sophia</namePart>
            <namePart type="family">Ananiadou</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hsin-Hsi</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hiroya</namePart>
            <namePart type="family">Takamura</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Chung-Chi</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-466-8</identifier>
    </relatedItem>
    <abstract>Evaluating causal mechanism explanations in generated financial news is challenging because stock price movements may have multiple plausible explanations without a unique gold explanation. We propose a reference-free LLM-as-a-Judge framework that evaluates explanations from three perspectives: Relevance, Completeness, and Economic Plausibility. We construct 200 articles from Chinese stock market data and compare Gemini 3.5 Flash, GPT-5.5, and Claude Opus 5 with human annotations under zero-shot and few-shot settings. Before aggregating the evaluation results, we applied a systematic matched-subset procedure to examine potential self-preference and generation-source bias. Based on the data examined in this study, these analyses found no statistically significant evidence of systematic bias, supporting the aggregation of all 200 articles for the overall agreement analysis. The results show moderate to high agreement with human evaluation, with Completeness generally showing higher agreement than Relevance. Few-shot prompting does not consistently improve agreement and generally leads to more conservative judgments. These findings demonstrate the potential of LLM-as-a-Judge for reference-free evaluation of causal mechanism explanations in financial news.</abstract>
    <identifier type="citekey">xie-etal-2026-preliminary</identifier>
    <location>
        <url>https://aclanthology.org/2026.finnlp-1.39/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>469</start>
            <end>485</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Preliminary Study on LLM-as-a-Judge Evaluation of Causal Mechanism Explanations in Financial News and its Bias Analysis
%A Xie, Yucheng
%A Kosai, Yurina
%A Tsuchida, Rikuto
%A Utsuro, Takehito
%Y Huang, Jimin
%Y Lee, Yongjae
%Y Ananiadou, Sophia
%Y Chen, Hsin-Hsi
%Y Takamura, Hiroya
%Y Chen, Chung-Chi
%S Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-466-8
%F xie-etal-2026-preliminary
%X Evaluating causal mechanism explanations in generated financial news is challenging because stock price movements may have multiple plausible explanations without a unique gold explanation. We propose a reference-free LLM-as-a-Judge framework that evaluates explanations from three perspectives: Relevance, Completeness, and Economic Plausibility. We construct 200 articles from Chinese stock market data and compare Gemini 3.5 Flash, GPT-5.5, and Claude Opus 5 with human annotations under zero-shot and few-shot settings. Before aggregating the evaluation results, we applied a systematic matched-subset procedure to examine potential self-preference and generation-source bias. Based on the data examined in this study, these analyses found no statistically significant evidence of systematic bias, supporting the aggregation of all 200 articles for the overall agreement analysis. The results show moderate to high agreement with human evaluation, with Completeness generally showing higher agreement than Relevance. Few-shot prompting does not consistently improve agreement and generally leads to more conservative judgments. These findings demonstrate the potential of LLM-as-a-Judge for reference-free evaluation of causal mechanism explanations in financial news.
%U https://aclanthology.org/2026.finnlp-1.39/
%P 469-485

Markdown (Informal)

[Preliminary Study on LLM-as-a-Judge Evaluation of Causal Mechanism Explanations in Financial News and its Bias Analysis](https://aclanthology.org/2026.finnlp-1.39/) (Xie et al., FinNLP 2026)

• Preliminary Study on LLM-as-a-Judge Evaluation of Causal Mechanism Explanations in Financial News and its Bias Analysis (Xie et al., FinNLP 2026)

ACL

• Yucheng Xie, Yurina Kosai, Rikuto Tsuchida, and Takehito Utsuro. 2026. Preliminary Study on LLM-as-a-Judge Evaluation of Causal Mechanism Explanations in Financial News and its Bias Analysis. In Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing, pages 469–485, Budapest, Hungary. Association for Computational Linguistics.