<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="singh-etal-2023-enhancing">
    <titleInfo>
        <title>Enhancing Textbooks with Visuals from the Web for Improved Learning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Janvijay</namePart>
        <namePart type="family">Singh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vilém</namePart>
        <namePart type="family">Zouhar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mrinmaya</namePart>
        <namePart type="family">Sachan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Textbooks are one of the main mediums for delivering high-quality education to students. In particular, explanatory and illustrative visuals play a key role in retention, comprehension and general transfer of knowledge. However, many textbooks lack these interesting visuals to support student learning. In this paper, we investigate the effectiveness of vision-language models to automatically enhance textbooks with images from the web. We collect a dataset of e-textbooks in the math, science, social science and business domains. We then set up a text-image matching task that involves retrieving and appropriately assigning web images to textbooks, which we frame as a matching optimization problem. Through a crowd-sourced evaluation, we verify that (1) while the original textbook images are rated higher, automatically assigned ones are not far behind, and (2) the precise formulation of the optimization problem matters. We release the dataset of textbooks with an associated image bank to inspire further research in this intersectional area of computer vision and NLP for education.</abstract>
    <identifier type="citekey">singh-etal-2023-enhancing</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.731</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.731</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>11931</start>
            <end>11944</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Enhancing Textbooks with Visuals from the Web for Improved Learning
%A Singh, Janvijay
%A Zouhar, Vilém
%A Sachan, Mrinmaya
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F singh-etal-2023-enhancing
%X Textbooks are one of the main mediums for delivering high-quality education to students. In particular, explanatory and illustrative visuals play a key role in retention, comprehension and general transfer of knowledge. However, many textbooks lack these interesting visuals to support student learning. In this paper, we investigate the effectiveness of vision-language models to automatically enhance textbooks with images from the web. We collect a dataset of e-textbooks in the math, science, social science and business domains. We then set up a text-image matching task that involves retrieving and appropriately assigning web images to textbooks, which we frame as a matching optimization problem. Through a crowd-sourced evaluation, we verify that (1) while the original textbook images are rated higher, automatically assigned ones are not far behind, and (2) the precise formulation of the optimization problem matters. We release the dataset of textbooks with an associated image bank to inspire further research in this intersectional area of computer vision and NLP for education.
%R 10.18653/v1/2023.emnlp-main.731
%U https://aclanthology.org/2023.emnlp-main.731
%U https://doi.org/10.18653/v1/2023.emnlp-main.731
%P 11931-11944

Markdown (Informal)

[Enhancing Textbooks with Visuals from the Web for Improved Learning](https://aclanthology.org/2023.emnlp-main.731) (Singh et al., EMNLP 2023)

• Enhancing Textbooks with Visuals from the Web for Improved Learning (Singh et al., EMNLP 2023)

ACL

• Janvijay Singh, Vilém Zouhar, and Mrinmaya Sachan. 2023. Enhancing Textbooks with Visuals from the Web for Improved Learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 11931–11944, Singapore. Association for Computational Linguistics.