<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kimura-suzuki-2026-text">
    <titleInfo>
        <title>Text-Only vs. Image-Aware VLM Judges for Manga Translation Evaluation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Subaru</namePart>
        <namePart type="family">Kimura</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jun</namePart>
        <namePart type="family">Suzuki</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Eleventh Conference on Machine Translation</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Barry</namePart>
            <namePart type="family">Haddow</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tom</namePart>
            <namePart type="family">Kocmi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Philipp</namePart>
            <namePart type="family">Koehn</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Christof</namePart>
            <namePart type="family">Monz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-474-3</identifier>
    </relatedItem>
    <abstract>In manga translation, visual context outside speech-bubble text, including speaker identities, referents, and scene tone, is depicted in page images, making it natural to refer to images during translation evaluation. However, most conventional manga translation evaluations rely solely on text-only automatic metrics, leaving the impact of image input insufficiently explored. We propose a practical validation framework that employs the same vision-language model (VLM) as both a text-only judge (TOJ) and an image-aware judge (IAJ) to investigate evaluation divergences associated with image-aware evaluation procedures. We use a Japanese-English manga dataset of official translations aligned at the page and speech-bubble levels. First, adding page images to TOJ enabled the primary VLM to detect more degraded translations. Second, we analyzed evaluation divergences between TOJ and IAJ, finding different error counts and error-category distributions. Human verification did not reveal a consistent preference between TOJ and IAJ.</abstract>
    <identifier type="citekey">kimura-suzuki-2026-text</identifier>
    <location>
        <url>https://aclanthology.org/2026.wmt-main.20/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>345</start>
            <end>367</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Text-Only vs. Image-Aware VLM Judges for Manga Translation Evaluation
%A Kimura, Subaru
%A Suzuki, Jun
%Y Haddow, Barry
%Y Kocmi, Tom
%Y Koehn, Philipp
%Y Monz, Christof
%S Proceedings of the Eleventh Conference on Machine Translation
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-474-3
%F kimura-suzuki-2026-text
%X In manga translation, visual context outside speech-bubble text, including speaker identities, referents, and scene tone, is depicted in page images, making it natural to refer to images during translation evaluation. However, most conventional manga translation evaluations rely solely on text-only automatic metrics, leaving the impact of image input insufficiently explored. We propose a practical validation framework that employs the same vision-language model (VLM) as both a text-only judge (TOJ) and an image-aware judge (IAJ) to investigate evaluation divergences associated with image-aware evaluation procedures. We use a Japanese-English manga dataset of official translations aligned at the page and speech-bubble levels. First, adding page images to TOJ enabled the primary VLM to detect more degraded translations. Second, we analyzed evaluation divergences between TOJ and IAJ, finding different error counts and error-category distributions. Human verification did not reveal a consistent preference between TOJ and IAJ.
%U https://aclanthology.org/2026.wmt-main.20/
%P 345-367

Markdown (Informal)

[Text-Only vs. Image-Aware VLM Judges for Manga Translation Evaluation](https://aclanthology.org/2026.wmt-main.20/) (Kimura & Suzuki, WMT 2026)

• Text-Only vs. Image-Aware VLM Judges for Manga Translation Evaluation (Kimura & Suzuki, WMT 2026)

ACL

• Subaru Kimura and Jun Suzuki. 2026. Text-Only vs. Image-Aware VLM Judges for Manga Translation Evaluation. In Proceedings of the Eleventh Conference on Machine Translation, pages 345–367, Budapest, Hungary. Association for Computational Linguistics.