<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="iwamoto-kanayama-2024-llm-neologism">
    <titleInfo>
        <title>LLM Neologism: Emergence of Mutated Characters due to Byte Encoding</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Ran</namePart>
        <namePart type="family">Iwamoto</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hiroshi</namePart>
        <namePart type="family">Kanayama</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 17th International Natural Language Generation Conference</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Saad</namePart>
            <namePart type="family">Mahamood</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nguyen</namePart>
            <namePart type="given">Le</namePart>
            <namePart type="family">Minh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Daphne</namePart>
            <namePart type="family">Ippolito</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Tokyo, Japan</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The process of language generation, which selects the most probable tokens one by one, may intrinsically result in output strings that humans never utter. We name this phenomenon “LLM neologism” and investigate it focusing on Japanese, Chinese, and Korean languages, where tokens can be smaller than characters. Our findings show that LLM neologism occurs through the combination of two high-frequency words with common tokens. We also clarify the cause of LLM neologism in the tokenization process with limited vocabularies. The results of this study provides important clues for better encoding of multibyte characters, aiming to prevent catastrophic results in AI-generated documents.</abstract>
    <identifier type="citekey">iwamoto-kanayama-2024-llm-neologism</identifier>
    <location>
        <url>https://aclanthology.org/2024.inlg-main.3</url>
    </location>
    <part>
        <date>2024-09</date>
        <extent unit="page">
            <start>24</start>
            <end>29</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T LLM Neologism: Emergence of Mutated Characters due to Byte Encoding
%A Iwamoto, Ran
%A Kanayama, Hiroshi
%Y Mahamood, Saad
%Y Minh, Nguyen Le
%Y Ippolito, Daphne
%S Proceedings of the 17th International Natural Language Generation Conference
%D 2024
%8 September
%I Association for Computational Linguistics
%C Tokyo, Japan
%F iwamoto-kanayama-2024-llm-neologism
%X The process of language generation, which selects the most probable tokens one by one, may intrinsically result in output strings that humans never utter. We name this phenomenon “LLM neologism” and investigate it focusing on Japanese, Chinese, and Korean languages, where tokens can be smaller than characters. Our findings show that LLM neologism occurs through the combination of two high-frequency words with common tokens. We also clarify the cause of LLM neologism in the tokenization process with limited vocabularies. The results of this study provides important clues for better encoding of multibyte characters, aiming to prevent catastrophic results in AI-generated documents.
%U https://aclanthology.org/2024.inlg-main.3
%P 24-29

Markdown (Informal)

[LLM Neologism: Emergence of Mutated Characters due to Byte Encoding](https://aclanthology.org/2024.inlg-main.3) (Iwamoto & Kanayama, INLG 2024)

• LLM Neologism: Emergence of Mutated Characters due to Byte Encoding (Iwamoto & Kanayama, INLG 2024)

ACL

• Ran Iwamoto and Hiroshi Kanayama. 2024. LLM Neologism: Emergence of Mutated Characters due to Byte Encoding. In Proceedings of the 17th International Natural Language Generation Conference, pages 24–29, Tokyo, Japan. Association for Computational Linguistics.