<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ai-huang-2024-zero">
    <titleInfo>
        <title>Zero-shot Cross-lingual Alignment for Embedding Initialization</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Xi</namePart>
        <namePart type="family">Ai</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Zhiyong</namePart>
        <namePart type="family">Huang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Lun-Wei</namePart>
            <namePart type="family">Ku</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andre</namePart>
            <namePart type="family">Martins</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>For multilingual training, we present CrossInit, an initialization method that initializes embeddings into similar geometrical structures across languages in an unsupervised manner. CrossInit leverages a common cognitive linguistic mechanism, Zipf’s law, which indicates that similar concepts across languages have similar word ranks or frequencies in their monolingual corpora. Instead of considering point-to-point alignments based on ranks, CrossInit considers the same span of consecutive ranks in each language as the Positive pairs for alignment, while others out of the span are used as Negative pairs. CrossInit then employs Contrastive Learning to iteratively refine randomly initialized embeddings for similar geometrical structures across languages. Our experiments on Unsupervised NMT, XNLI, and MLQA showed significant gains in low-resource and dissimilar languages after applying CrossInit.</abstract>
    <identifier type="citekey">ai-huang-2024-zero</identifier>
    <identifier type="doi">10.18653/v1/2024.findings-acl.358</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-acl.358/</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>5997</start>
            <end>6007</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Zero-shot Cross-lingual Alignment for Embedding Initialization
%A Ai, Xi
%A Huang, Zhiyong
%Y Ku, Lun-Wei
%Y Martins, Andre
%Y Srikumar, Vivek
%S Findings of the Association for Computational Linguistics: ACL 2024
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F ai-huang-2024-zero
%X For multilingual training, we present CrossInit, an initialization method that initializes embeddings into similar geometrical structures across languages in an unsupervised manner. CrossInit leverages a common cognitive linguistic mechanism, Zipf’s law, which indicates that similar concepts across languages have similar word ranks or frequencies in their monolingual corpora. Instead of considering point-to-point alignments based on ranks, CrossInit considers the same span of consecutive ranks in each language as the Positive pairs for alignment, while others out of the span are used as Negative pairs. CrossInit then employs Contrastive Learning to iteratively refine randomly initialized embeddings for similar geometrical structures across languages. Our experiments on Unsupervised NMT, XNLI, and MLQA showed significant gains in low-resource and dissimilar languages after applying CrossInit.
%R 10.18653/v1/2024.findings-acl.358
%U https://aclanthology.org/2024.findings-acl.358/
%U https://doi.org/10.18653/v1/2024.findings-acl.358
%P 5997-6007

Markdown (Informal)

[Zero-shot Cross-lingual Alignment for Embedding Initialization](https://aclanthology.org/2024.findings-acl.358/) (Ai & Huang, Findings 2024)

• Zero-shot Cross-lingual Alignment for Embedding Initialization (Ai & Huang, Findings 2024)

ACL

• Xi Ai and Zhiyong Huang. 2024. Zero-shot Cross-lingual Alignment for Embedding Initialization. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5997–6007, Bangkok, Thailand. Association for Computational Linguistics.