<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zhang-etal-2026-learning-concepts">
    <titleInfo>
        <title>Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Christine</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dan</namePart>
        <namePart type="family">Jurafsky</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chen</namePart>
        <namePart type="family">Shani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>The next-token prediction (NTP) objective trains language models to predict a single token at each step, even though many continuations can express the same meaning. For example, in the sentence “this sticker can be placed here”, positioned, attached, or put are all plausible alternatives. While standard NTP training treats these alternatives as mutually exclusive targets, we explore a self-supervised framework that encourages models to predict concepts, approximated as sets of semantically equivalent tokens. Models trained with this concept supervision align better with human similarity judgments, improve classification, clustering, and reranking performance, and achieve comparable or stronger downstream reasoning. These gains come with lower perplexity on semantically meaningful words (Section 3.2) and only minimal increases in global perplexity, suggesting that concepts enhance semantic alignment while preserving language modeling quality.</abstract>
    <identifier type="citekey">zhang-etal-2026-learning-concepts</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.1709/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>38927</start>
            <end>38956</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models
%A Zhang, Christine
%A Jurafsky, Dan
%A Shani, Chen
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F zhang-etal-2026-learning-concepts
%X The next-token prediction (NTP) objective trains language models to predict a single token at each step, even though many continuations can express the same meaning. For example, in the sentence “this sticker can be placed here”, positioned, attached, or put are all plausible alternatives. While standard NTP training treats these alternatives as mutually exclusive targets, we explore a self-supervised framework that encourages models to predict concepts, approximated as sets of semantically equivalent tokens. Models trained with this concept supervision align better with human similarity judgments, improve classification, clustering, and reranking performance, and achieve comparable or stronger downstream reasoning. These gains come with lower perplexity on semantically meaningful words (Section 3.2) and only minimal increases in global perplexity, suggesting that concepts enhance semantic alignment while preserving language modeling quality.
%U https://aclanthology.org/2026.emnlp-main.1709/
%P 38927-38956

Markdown (Informal)

[Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models](https://aclanthology.org/2026.emnlp-main.1709/) (Zhang et al., EMNLP 2026)

• Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models (Zhang et al., EMNLP 2026)

ACL

• Christine Zhang, Dan Jurafsky, and Chen Shani. 2026. Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 38927–38956, Budapest, Hungary. Association for Computational Linguistics.