@inproceedings{rapolu-2026-fever,
title = "When {FEVER} Meets Finance: A Controlled Benchmark Reveals Overlap-Driven Financial Claim Verification",
author = "Rapolu, Suprabhat",
editor = "Huang, Jimin and
Lee, Yongjae and
Ananiadou, Sophia and
Chen, Hsin-Hsi and
Takamura, Hiroya and
Chen, Chung-Chi",
booktitle = "Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing",
month = oct,
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.finnlp-1.42/",
pages = "518--523",
ISBN = "979-8-89176-466-8",
abstract = "Realistic financial misinformation is typically a single-token edit of a true statement, yet no public financial dataset supports per-edit statements about which factual changes a verification system can detect. We introduce a controlled benchmark of 1,489 claim-evidence pairs derived from SEC 10-K filings via documented FactCC-style perturbations (number changes, negation flips, entity swaps, evidence mismatches), each pair tagged with its generation rule, and use it to evaluate eleven models spanning three technological generations - TF-IDF linear classifiers, tree ensembles, and neural models including a fine-tuned transformer - all trained on FEVER under one protocol. Per-edit attribution is our central result: the strongest model verifies verbatim financial claims at 98.8{\%} accuracy but detects single-token falsifications at 33{\%} accuracy or below, with number changes and negation flips detected at similar rates and entity swaps far the hardest (2.6{\%}, on 38 pairs). Two diagnoses follow. A hypothesis-only probe shows the classical tier{'}s in-domain competence is claim-surface artifact - a claim-only classifier reaches macro-F1 0.556 against 0.549 for full claim+evidence models - while our benchmark shows no such artifact under the same probe (0.358). And transfer to real financial claims collapses: on Fin-Fact, no learned model beats a tuned lexical-overlap heuristic (macro-F1 0.368), despite in-domain scores up to 0.841. We release the benchmark, generation code, and evaluation harness."
}<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="rapolu-2026-fever">
<titleInfo>
<title>When FEVER Meets Finance: A Controlled Benchmark Reveals Overlap-Driven Financial Claim Verification</title>
</titleInfo>
<name type="personal">
<namePart type="given">Suprabhat</namePart>
<namePart type="family">Rapolu</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<originInfo>
<dateIssued>2026-10</dateIssued>
</originInfo>
<typeOfResource>text</typeOfResource>
<relatedItem type="host">
<titleInfo>
<title>Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing</title>
</titleInfo>
<name type="personal">
<namePart type="given">Jimin</namePart>
<namePart type="family">Huang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yongjae</namePart>
<namePart type="family">Lee</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Sophia</namePart>
<namePart type="family">Ananiadou</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hsin-Hsi</namePart>
<namePart type="family">Chen</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hiroya</namePart>
<namePart type="family">Takamura</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Chung-Chi</namePart>
<namePart type="family">Chen</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<originInfo>
<publisher>Association for Computational Linguistics</publisher>
<place>
<placeTerm type="text">Budapest, Hungary</placeTerm>
</place>
</originInfo>
<genre authority="marcgt">conference publication</genre>
<identifier type="isbn">979-8-89176-466-8</identifier>
</relatedItem>
<abstract>Realistic financial misinformation is typically a single-token edit of a true statement, yet no public financial dataset supports per-edit statements about which factual changes a verification system can detect. We introduce a controlled benchmark of 1,489 claim-evidence pairs derived from SEC 10-K filings via documented FactCC-style perturbations (number changes, negation flips, entity swaps, evidence mismatches), each pair tagged with its generation rule, and use it to evaluate eleven models spanning three technological generations - TF-IDF linear classifiers, tree ensembles, and neural models including a fine-tuned transformer - all trained on FEVER under one protocol. Per-edit attribution is our central result: the strongest model verifies verbatim financial claims at 98.8% accuracy but detects single-token falsifications at 33% accuracy or below, with number changes and negation flips detected at similar rates and entity swaps far the hardest (2.6%, on 38 pairs). Two diagnoses follow. A hypothesis-only probe shows the classical tier’s in-domain competence is claim-surface artifact - a claim-only classifier reaches macro-F1 0.556 against 0.549 for full claim+evidence models - while our benchmark shows no such artifact under the same probe (0.358). And transfer to real financial claims collapses: on Fin-Fact, no learned model beats a tuned lexical-overlap heuristic (macro-F1 0.368), despite in-domain scores up to 0.841. We release the benchmark, generation code, and evaluation harness.</abstract>
<identifier type="citekey">rapolu-2026-fever</identifier>
<location>
<url>https://aclanthology.org/2026.finnlp-1.42/</url>
</location>
<part>
<date>2026-10</date>
<extent unit="page">
<start>518</start>
<end>523</end>
</extent>
</part>
</mods>
</modsCollection>
%0 Conference Proceedings
%T When FEVER Meets Finance: A Controlled Benchmark Reveals Overlap-Driven Financial Claim Verification
%A Rapolu, Suprabhat
%Y Huang, Jimin
%Y Lee, Yongjae
%Y Ananiadou, Sophia
%Y Chen, Hsin-Hsi
%Y Takamura, Hiroya
%Y Chen, Chung-Chi
%S Proceedings of the 11th Workshop on Financial Technology and Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-466-8
%F rapolu-2026-fever
%X Realistic financial misinformation is typically a single-token edit of a true statement, yet no public financial dataset supports per-edit statements about which factual changes a verification system can detect. We introduce a controlled benchmark of 1,489 claim-evidence pairs derived from SEC 10-K filings via documented FactCC-style perturbations (number changes, negation flips, entity swaps, evidence mismatches), each pair tagged with its generation rule, and use it to evaluate eleven models spanning three technological generations - TF-IDF linear classifiers, tree ensembles, and neural models including a fine-tuned transformer - all trained on FEVER under one protocol. Per-edit attribution is our central result: the strongest model verifies verbatim financial claims at 98.8% accuracy but detects single-token falsifications at 33% accuracy or below, with number changes and negation flips detected at similar rates and entity swaps far the hardest (2.6%, on 38 pairs). Two diagnoses follow. A hypothesis-only probe shows the classical tier’s in-domain competence is claim-surface artifact - a claim-only classifier reaches macro-F1 0.556 against 0.549 for full claim+evidence models - while our benchmark shows no such artifact under the same probe (0.358). And transfer to real financial claims collapses: on Fin-Fact, no learned model beats a tuned lexical-overlap heuristic (macro-F1 0.368), despite in-domain scores up to 0.841. We release the benchmark, generation code, and evaluation harness.
%U https://aclanthology.org/2026.finnlp-1.42/
%P 518-523
Markdown (Informal)
[When FEVER Meets Finance: A Controlled Benchmark Reveals Overlap-Driven Financial Claim Verification](https://aclanthology.org/2026.finnlp-1.42/) (Rapolu, FinNLP 2026)
ACL