<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lee-etal-2026-vlms">
    <titleInfo>
        <title>Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Gwang</namePart>
        <namePart type="given">Gook</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kenan</namePart>
        <namePart type="given">E</namePart>
        <namePart type="family">Ak</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jay</namePart>
        <namePart type="family">Mohta</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yan</namePart>
        <namePart type="family">Xu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dimitrios</namePart>
        <namePart type="family">Dimitriadis</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-462-0</identifier>
    </relatedItem>
    <abstract>Vision Language Models (VLMs) are increasingly used in place of traditional OCR pipelines for document understanding. We show that VLMs do not always transcribe text literally. When the input contains imperfect text, they often rewrite it into a more plausible form, a behavior that clean-text OCR benchmarks do not detect. We introduce FaithC4, a multilingual perturbation benchmark of 1,455 single-page documents (English, Chinese, Korean) with three perturbation families: scramble, random substitution, and visually similar substitution. We use it to evaluate 15 systems spanning general-purpose VLMs, OCR-specialized VLMs, and traditional OCR pipelines. Traditional OCR is near-perfect on perturbed text, OCR-specialized VLMs degrade by 0.2–2.3 points of Word Error Rate (WER), and general-purpose VLMs degrade by up to 4.5 points in English language. To explain this gap, we probe Qwen3-VL-4B layer-by-layer and find that rewriting occurs when a perturbed word’s final-layer FFN representation stays close to the original word’s encoding, while faithful transcription occurs when the representation is sufficiently disrupted. Shorter words give the model less evidence to disrupt the representation and are therefore rewritten far more often than longer ones, with 4-character words rewritten more than ten times as often as words of 8 or more characters. We will release FaithC4 and our evaluation code upon acceptance to support future research on transcription faithfulness.</abstract>
    <identifier type="citekey">lee-etal-2026-vlms</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-emnlp.1368/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>29077</start>
            <end>29092</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models
%A Lee, Gwang Gook
%A Ak, Kenan E.
%A Mohta, Jay
%A Xu, Yan
%A Dimitriadis, Dimitrios
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Findings of the Association for Computational Linguistics: EMNLP 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-462-0
%F lee-etal-2026-vlms
%X Vision Language Models (VLMs) are increasingly used in place of traditional OCR pipelines for document understanding. We show that VLMs do not always transcribe text literally. When the input contains imperfect text, they often rewrite it into a more plausible form, a behavior that clean-text OCR benchmarks do not detect. We introduce FaithC4, a multilingual perturbation benchmark of 1,455 single-page documents (English, Chinese, Korean) with three perturbation families: scramble, random substitution, and visually similar substitution. We use it to evaluate 15 systems spanning general-purpose VLMs, OCR-specialized VLMs, and traditional OCR pipelines. Traditional OCR is near-perfect on perturbed text, OCR-specialized VLMs degrade by 0.2–2.3 points of Word Error Rate (WER), and general-purpose VLMs degrade by up to 4.5 points in English language. To explain this gap, we probe Qwen3-VL-4B layer-by-layer and find that rewriting occurs when a perturbed word’s final-layer FFN representation stays close to the original word’s encoding, while faithful transcription occurs when the representation is sufficiently disrupted. Shorter words give the model less evidence to disrupt the representation and are therefore rewritten far more often than longer ones, with 4-character words rewritten more than ten times as often as words of 8 or more characters. We will release FaithC4 and our evaluation code upon acceptance to support future research on transcription faithfulness.
%U https://aclanthology.org/2026.findings-emnlp.1368/
%P 29077-29092

Markdown (Informal)

[Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models](https://aclanthology.org/2026.findings-emnlp.1368/) (Lee et al., Findings 2026)

• Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models (Lee et al., Findings 2026)

ACL

• Gwang Gook Lee, Kenan E. Ak, Jay Mohta, Yan Xu, and Dimitrios Dimitriadis. 2026. Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 29077–29092, Budapest, Hungary. Association for Computational Linguistics.