<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="bonnier-2024-revisiting">
    <titleInfo>
        <title>Revisiting Multimodal Transformers for Tabular Data with Text Fields</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Thomas</namePart>
        <namePart type="family">Bonnier</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Lun-Wei</namePart>
            <namePart type="family">Ku</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andre</namePart>
            <namePart type="family">Martins</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Tabular data with text fields can be leveraged in applications such as financial risk assessment or medical diagnosis prediction. When employing multimodal approaches to make predictions based on these modalities, it is crucial to make the most appropriate modeling choices in terms of numerical feature encoding or fusion strategy. In this paper, we focus on multimodal classification tasks based on tabular datasets with text fields. We build on multimodal Transformers to propose the Tabular-Text Transformer (TTT), a tabular/text dual-stream Transformer network. This architecture includes a distance-to-quantile embedding scheme for numerical features and an overall attention module which concurrently considers self-attention and cross-modal attention. Further, we leverage the two well-informed modality streams to estimate whether a prediction is uncertain or not. To explain uncertainty in terms of feature values, we use a sampling-based approximation of Shapley values in a bimodal context, with two options for the value function. To show the efficacy and relevance of this approach, we compare it to six baselines and measure its ability to quantify and explain uncertainty against various methods. Our code is available at https://github.com/thomas-bonnier/TabularTextTransformer.</abstract>
    <identifier type="citekey">bonnier-2024-revisiting</identifier>
    <identifier type="doi">10.18653/v1/2024.findings-acl.87</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-acl.87</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>1481</start>
            <end>1500</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Revisiting Multimodal Transformers for Tabular Data with Text Fields
%A Bonnier, Thomas
%Y Ku, Lun-Wei
%Y Martins, Andre
%Y Srikumar, Vivek
%S Findings of the Association for Computational Linguistics: ACL 2024
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F bonnier-2024-revisiting
%X Tabular data with text fields can be leveraged in applications such as financial risk assessment or medical diagnosis prediction. When employing multimodal approaches to make predictions based on these modalities, it is crucial to make the most appropriate modeling choices in terms of numerical feature encoding or fusion strategy. In this paper, we focus on multimodal classification tasks based on tabular datasets with text fields. We build on multimodal Transformers to propose the Tabular-Text Transformer (TTT), a tabular/text dual-stream Transformer network. This architecture includes a distance-to-quantile embedding scheme for numerical features and an overall attention module which concurrently considers self-attention and cross-modal attention. Further, we leverage the two well-informed modality streams to estimate whether a prediction is uncertain or not. To explain uncertainty in terms of feature values, we use a sampling-based approximation of Shapley values in a bimodal context, with two options for the value function. To show the efficacy and relevance of this approach, we compare it to six baselines and measure its ability to quantify and explain uncertainty against various methods. Our code is available at https://github.com/thomas-bonnier/TabularTextTransformer.
%R 10.18653/v1/2024.findings-acl.87
%U https://aclanthology.org/2024.findings-acl.87
%U https://doi.org/10.18653/v1/2024.findings-acl.87
%P 1481-1500

Markdown (Informal)

[Revisiting Multimodal Transformers for Tabular Data with Text Fields](https://aclanthology.org/2024.findings-acl.87) (Bonnier, Findings 2024)

• Revisiting Multimodal Transformers for Tabular Data with Text Fields (Bonnier, Findings 2024)

ACL

• Thomas Bonnier. 2024. Revisiting Multimodal Transformers for Tabular Data with Text Fields. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1481–1500, Bangkok, Thailand. Association for Computational Linguistics.