<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="harish-2026-document">
    <titleInfo>
        <title>Most of a Document-VQA Score Is Answer Formatting</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Srinivas</namePart>
        <namePart type="family">Harish</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>The Proceedings of Workshop on Document Intelligence and Understanding</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Santosh</namePart>
            <namePart type="family">T.Y.S.S.</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Amit</namePart>
            <namePart type="family">Agarwal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Gupta</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yova</namePart>
            <namePart type="family">Kementchedjhieva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Aparna</namePart>
            <namePart type="family">Garimella</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yufang</namePart>
            <namePart type="family">Hou</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-477-4</identifier>
    </relatedItem>
    <abstract>DocVQA and InfographicVQA are scored with average normalized Levenshtein similarity (ANLS), and a DocVQA number is read as a measure of how well a model reads a page, but the lmms-eval and VLMEvalKit harnesses that compute it append a one-line terse-answer suffix to every question. ANLS divides edit distance by the length of the longer string, so a correct answer inside a sentence more than twice its length falls below the 0.5 threshold and scores zero, and removing only the suffix moved Qwen2.5-VL-3B’s DocVQA ANLS from 94.2 to 7.5. On that model ANLS given a short answer (within two tokens of the longest gold) stayed between 93.9 and 95.7 across three suffixes whose scores spanned 94.2 to 65.3, so the reported score followed the share of short answers more than the share that contained the gold. Two suffixes that lmms-eval defines for the same task differed by 28.8 points, and under one of them 79% of the answers ANLS scored zero contained the gold answer, all 305 of which a judge with negated and two-candidate controls accepted as the committed answer. On a 317-question subset, four hosted models lost between 34.4 and 89.5 points without the suffix (Gemini 3.8 Flash fell from 92.0 to 2.5), and with the suffix in place, deleting Markdown emphasis markers raised Claude Opus 5.5 by 10.9 points. On Qwen2.5-VL-3B a length-gated span scorer moved 13.3 points across the five suffixes where ANLS moved 86.6, and it scored a submission of the page’s whole OCR at 1.3, where best-subsequence exact match and the learned PEDANTS classifier gave 93.0 and 98.6. On ChartQA the suffix was worth 28.4 points of relaxed accuracy to Qwen2.5-VL-3B. We release the scorer and recommend reporting the prompt suffix and output post-processing with every document-VQA score.</abstract>
    <identifier type="citekey">harish-2026-document</identifier>
    <location>
        <url>https://aclanthology.org/2026.docinsights-1.6/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>73</start>
            <end>80</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Most of a Document-VQA Score Is Answer Formatting
%A Harish, Srinivas
%Y T.Y.S.S., Santosh
%Y Agarwal, Amit
%Y Gupta, Vivek
%Y Kementchedjhieva, Yova
%Y Garimella, Aparna
%Y Hou, Yufang
%S The Proceedings of Workshop on Document Intelligence and Understanding
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-477-4
%F harish-2026-document
%X DocVQA and InfographicVQA are scored with average normalized Levenshtein similarity (ANLS), and a DocVQA number is read as a measure of how well a model reads a page, but the lmms-eval and VLMEvalKit harnesses that compute it append a one-line terse-answer suffix to every question. ANLS divides edit distance by the length of the longer string, so a correct answer inside a sentence more than twice its length falls below the 0.5 threshold and scores zero, and removing only the suffix moved Qwen2.5-VL-3B’s DocVQA ANLS from 94.2 to 7.5. On that model ANLS given a short answer (within two tokens of the longest gold) stayed between 93.9 and 95.7 across three suffixes whose scores spanned 94.2 to 65.3, so the reported score followed the share of short answers more than the share that contained the gold. Two suffixes that lmms-eval defines for the same task differed by 28.8 points, and under one of them 79% of the answers ANLS scored zero contained the gold answer, all 305 of which a judge with negated and two-candidate controls accepted as the committed answer. On a 317-question subset, four hosted models lost between 34.4 and 89.5 points without the suffix (Gemini 3.8 Flash fell from 92.0 to 2.5), and with the suffix in place, deleting Markdown emphasis markers raised Claude Opus 5.5 by 10.9 points. On Qwen2.5-VL-3B a length-gated span scorer moved 13.3 points across the five suffixes where ANLS moved 86.6, and it scored a submission of the page’s whole OCR at 1.3, where best-subsequence exact match and the learned PEDANTS classifier gave 93.0 and 98.6. On ChartQA the suffix was worth 28.4 points of relaxed accuracy to Qwen2.5-VL-3B. We release the scorer and recommend reporting the prompt suffix and output post-processing with every document-VQA score.
%U https://aclanthology.org/2026.docinsights-1.6/
%P 73-80

Markdown (Informal)

[Most of a Document-VQA Score Is Answer Formatting](https://aclanthology.org/2026.docinsights-1.6/) (Harish, DocInsights 2026)

• Most of a Document-VQA Score Is Answer Formatting (Harish, DocInsights 2026)

ACL

• Srinivas Harish. 2026. Most of a Document-VQA Score Is Answer Formatting. In The Proceedings of Workshop on Document Intelligence and Understanding, pages 73–80, Budapest, Hungary. Association for Computational Linguistics.