<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="goyal-2026-expense">
    <titleInfo>
        <title>The Expense of Seeing: Attaining Trustworthy Multimodal Reasoning Within the Monolithic Paradigm</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Karan</namePart>
        <namePart type="family">Goyal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ruifeng</namePart>
            <namePart type="family">Xu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ayu</namePart>
            <namePart type="family">Purwarianti</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tong</namePart>
            <namePart type="family">Rong</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jun</namePart>
            <namePart type="family">Suzuki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hengqin, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-439-2</identifier>
    </relatedItem>
    <abstract>The rapid proliferation of Vision-Language Models (VLMs) is often framed as enabling unified multimodal understanding, but rests on an under-examined assumption: that current VLMs faithfully synthesise multimodal data. We argue they often do not, and that this gap reflects a trustworthiness problem in the dominant Vision Encoder-Projector-LLM paradigm. Rather than extracting grounded knowledge from visual inputs, state-of-the-art models frequently exhibit functional blindness, exploiting strong language priors to bypass severe visual representation bottlenecks. We challenge the conventional methodology of multimodal evaluation, which relies on data ablation or new dataset creation and therefore conflates dataset biases with architectural incapacity. We propose an information-theoretic departure: the *Modality Translation Protocol*, which preserves a sample’s semantic payload while translating its modality rather than ablating it. From this, we derive three metrics—the Toll (*ToS*), Curse (*CoS*), and Fallacy (*FoS*) of Seeing—culminating in the *Semantic Sufficiency Criterion (SSC)*, a single condition that detects, quantifies, and localises multimodal failures to the visual encoder or the projection head. We further hypothesise a *Divergence Law of Multimodal Scaling*: as language backbones grow, the penalty of the visual bottleneck may widen rather than close. Examining the closest-matching existing benchmarks, motivates our central call: the construction, through *Semantic Equivalence Engineering*, of a large, diverse dataset whose modalities are isomorphic by construction. We frame this not as an experiment awaiting compute, but as a mission for the community: to move beyond “multimodal gain” and build the substrate on which faithful multimodal architectures can be designed.</abstract>
    <identifier type="citekey">goyal-2026-expense</identifier>
    <location>
        <url>https://aclanthology.org/2026.aacl-long.251/</url>
    </location>
    <part>
        <date>2026-11</date>
        <extent unit="page">
            <start>5604</start>
            <end>5614</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The Expense of Seeing: Attaining Trustworthy Multimodal Reasoning Within the Monolithic Paradigm
%A Goyal, Karan
%Y Zhang, Yue
%Y Xu, Ruifeng
%Y Purwarianti, Ayu
%Y Rong, Tong
%Y Suzuki, Jun
%S Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2026
%8 November
%I Association for Computational Linguistics
%C Hengqin, China
%@ 979-8-89176-439-2
%F goyal-2026-expense
%X The rapid proliferation of Vision-Language Models (VLMs) is often framed as enabling unified multimodal understanding, but rests on an under-examined assumption: that current VLMs faithfully synthesise multimodal data. We argue they often do not, and that this gap reflects a trustworthiness problem in the dominant Vision Encoder-Projector-LLM paradigm. Rather than extracting grounded knowledge from visual inputs, state-of-the-art models frequently exhibit functional blindness, exploiting strong language priors to bypass severe visual representation bottlenecks. We challenge the conventional methodology of multimodal evaluation, which relies on data ablation or new dataset creation and therefore conflates dataset biases with architectural incapacity. We propose an information-theoretic departure: the *Modality Translation Protocol*, which preserves a sample’s semantic payload while translating its modality rather than ablating it. From this, we derive three metrics—the Toll (*ToS*), Curse (*CoS*), and Fallacy (*FoS*) of Seeing—culminating in the *Semantic Sufficiency Criterion (SSC)*, a single condition that detects, quantifies, and localises multimodal failures to the visual encoder or the projection head. We further hypothesise a *Divergence Law of Multimodal Scaling*: as language backbones grow, the penalty of the visual bottleneck may widen rather than close. Examining the closest-matching existing benchmarks, motivates our central call: the construction, through *Semantic Equivalence Engineering*, of a large, diverse dataset whose modalities are isomorphic by construction. We frame this not as an experiment awaiting compute, but as a mission for the community: to move beyond “multimodal gain” and build the substrate on which faithful multimodal architectures can be designed.
%U https://aclanthology.org/2026.aacl-long.251/
%P 5604-5614

Markdown (Informal)

[The Expense of Seeing: Attaining Trustworthy Multimodal Reasoning Within the Monolithic Paradigm](https://aclanthology.org/2026.aacl-long.251/) (Goyal, AACL-IJCNLP 2026)

• The Expense of Seeing: Attaining Trustworthy Multimodal Reasoning Within the Monolithic Paradigm (Goyal, AACL-IJCNLP 2026)

ACL

• Karan Goyal. 2026. The Expense of Seeing: Attaining Trustworthy Multimodal Reasoning Within the Monolithic Paradigm. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5604–5614, Hengqin, China. Association for Computational Linguistics.