<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="karnysheva-etal-2026-probing">
    <titleInfo>
        <title>Probing Chemical Language Models: Effects of Pre-training and Fine-tuning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Anna</namePart>
        <namePart type="family">Karnysheva</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dietrich</namePart>
        <namePart type="family">Klakow</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ji-Ung</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Chemical language models (CLMs) are trained with linearized representations such as SMILES, yet it remains unclear which chemically meaningful substructures they encode. To foster a better understanding of CLMs, we conduct a systematic study and probe for 78 molecular substructures across eight pre-trained and six randomly initialized models. We furthermore study how fine-tuning on chemical downstream tasks affects the learned representations of molecular substructures. Our results show that pre-training generally improves molecular structure awareness of CLMs, particularly in the upper layers. Moreover, randomly initialized models already encode ring structures well in the first layer. Our analysis on two chemical downstream tasks further reveals that, interestingly, fine-tuning affects task-relevant molecular substructures more than others, indicating that the changes in the representations follow chemical theory.</abstract>
    <identifier type="citekey">karnysheva-etal-2026-probing</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.2662/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>60585</start>
            <end>60627</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Probing Chemical Language Models: Effects of Pre-training and Fine-tuning
%A Karnysheva, Anna
%A Klakow, Dietrich
%A Lee, Ji-Ung
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F karnysheva-etal-2026-probing
%X Chemical language models (CLMs) are trained with linearized representations such as SMILES, yet it remains unclear which chemically meaningful substructures they encode. To foster a better understanding of CLMs, we conduct a systematic study and probe for 78 molecular substructures across eight pre-trained and six randomly initialized models. We furthermore study how fine-tuning on chemical downstream tasks affects the learned representations of molecular substructures. Our results show that pre-training generally improves molecular structure awareness of CLMs, particularly in the upper layers. Moreover, randomly initialized models already encode ring structures well in the first layer. Our analysis on two chemical downstream tasks further reveals that, interestingly, fine-tuning affects task-relevant molecular substructures more than others, indicating that the changes in the representations follow chemical theory.
%U https://aclanthology.org/2026.emnlp-main.2662/
%P 60585-60627

Markdown (Informal)

[Probing Chemical Language Models: Effects of Pre-training and Fine-tuning](https://aclanthology.org/2026.emnlp-main.2662/) (Karnysheva et al., EMNLP 2026)

• Probing Chemical Language Models: Effects of Pre-training and Fine-tuning (Karnysheva et al., EMNLP 2026)

ACL

• Anna Karnysheva, Dietrich Klakow, and Ji-Ung Lee. 2026. Probing Chemical Language Models: Effects of Pre-training and Fine-tuning. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 60585–60627, Budapest, Hungary. Association for Computational Linguistics.