<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="qorib-etal-2024-decoder">
    <titleInfo>
        <title>Are Decoder-Only Language Models Better than Encoder-Only Language Models in Understanding Word Meaning?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Muhammad</namePart>
        <namePart type="family">Qorib</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Geonsik</namePart>
        <namePart type="family">Moon</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hwee</namePart>
        <namePart type="given">Tou</namePart>
        <namePart type="family">Ng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Lun-Wei</namePart>
            <namePart type="family">Ku</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andre</namePart>
            <namePart type="family">Martins</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The natural language processing field has been evolving around language models for the past few years, from the usage of n-gram language models for re-ranking, to transfer learning with encoder-only (BERT-like) language models, and finally to large language models (LLMs) as general solvers. LLMs are dominated by the decoder-only type, and they are popular for their efficacy in numerous tasks. LLMs are regarded as having strong comprehension abilities and strong capabilities to solve new unseen tasks. As such, people may quickly assume that decoder-only LLMs always perform better than the encoder-only ones, especially for understanding word meaning. In this paper, we demonstrate that decoder-only LLMs perform worse on word meaning comprehension than an encoder-only language model that has vastly fewer parameters.</abstract>
    <identifier type="citekey">qorib-etal-2024-decoder</identifier>
    <identifier type="doi">10.18653/v1/2024.findings-acl.967</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-acl.967</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>16339</start>
            <end>16347</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Are Decoder-Only Language Models Better than Encoder-Only Language Models in Understanding Word Meaning?
%A Qorib, Muhammad
%A Moon, Geonsik
%A Ng, Hwee Tou
%Y Ku, Lun-Wei
%Y Martins, Andre
%Y Srikumar, Vivek
%S Findings of the Association for Computational Linguistics: ACL 2024
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F qorib-etal-2024-decoder
%X The natural language processing field has been evolving around language models for the past few years, from the usage of n-gram language models for re-ranking, to transfer learning with encoder-only (BERT-like) language models, and finally to large language models (LLMs) as general solvers. LLMs are dominated by the decoder-only type, and they are popular for their efficacy in numerous tasks. LLMs are regarded as having strong comprehension abilities and strong capabilities to solve new unseen tasks. As such, people may quickly assume that decoder-only LLMs always perform better than the encoder-only ones, especially for understanding word meaning. In this paper, we demonstrate that decoder-only LLMs perform worse on word meaning comprehension than an encoder-only language model that has vastly fewer parameters.
%R 10.18653/v1/2024.findings-acl.967
%U https://aclanthology.org/2024.findings-acl.967
%U https://doi.org/10.18653/v1/2024.findings-acl.967
%P 16339-16347

Markdown (Informal)

[Are Decoder-Only Language Models Better than Encoder-Only Language Models in Understanding Word Meaning?](https://aclanthology.org/2024.findings-acl.967) (Qorib et al., Findings 2024)

• Are Decoder-Only Language Models Better than Encoder-Only Language Models in Understanding Word Meaning? (Qorib et al., Findings 2024)

ACL

• Muhammad Qorib, Geonsik Moon, and Hwee Tou Ng. 2024. Are Decoder-Only Language Models Better than Encoder-Only Language Models in Understanding Word Meaning?. In Findings of the Association for Computational Linguistics: ACL 2024, pages 16339–16347, Bangkok, Thailand. Association for Computational Linguistics.