<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="moghe-etal-2021-cross">
    <titleInfo>
        <title>Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Nikita</namePart>
        <namePart type="family">Moghe</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mark</namePart>
        <namePart type="family">Steedman</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Alexandra</namePart>
        <namePart type="family">Birch</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online and Punta Cana, Dominican Republic</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recent progress in task-oriented neural dialogue systems is largely focused on a handful of languages, as annotation of training data is tedious and expensive. Machine translation has been used to make systems multilingual, but this can introduce a pipeline of errors. Another promising solution is using cross-lingual transfer learning through pretrained multilingual models. Existing methods train multilingual models with additional code-mixed task data or refine the cross-lingual representations through parallel ontologies. In this work, we enhance the transfer learning process by intermediate fine-tuning of pretrained multilingual models, where the multilingual models are fine-tuned with different but related data and/or tasks. Specifically, we use parallel and conversational movie subtitles datasets to design cross-lingual intermediate tasks suitable for downstream dialogue tasks. We use only 200K lines of parallel data for intermediate fine-tuning which is already available for 1782 language pairs. We test our approach on the cross-lingual dialogue state tracking task for the parallel MultiWoZ (English -\textgreater Chinese, Chinese -\textgreater English) and Multilingual WoZ (English -\textgreater German, English -\textgreater Italian) datasets. We achieve impressive improvements (\textgreater 20% on joint goal accuracy) on the parallel MultiWoZ dataset and the Multilingual WoZ dataset over the vanilla baseline with only 10% of the target language task data and zero-shot setup respectively.</abstract>
    <identifier type="citekey">moghe-etal-2021-cross</identifier>
    <identifier type="doi">10.18653/v1/2021.emnlp-main.87</identifier>
    <location>
        <url>https://aclanthology.org/2021.emnlp-main.87</url>
    </location>
    <part>
        <date>2021-11</date>
        <extent unit="page">
            <start>1137</start>
            <end>1150</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking
%A Moghe, Nikita
%A Steedman, Mark
%A Birch, Alexandra
%S Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
%D 2021
%8 November
%I Association for Computational Linguistics
%C Online and Punta Cana, Dominican Republic
%F moghe-etal-2021-cross
%X Recent progress in task-oriented neural dialogue systems is largely focused on a handful of languages, as annotation of training data is tedious and expensive. Machine translation has been used to make systems multilingual, but this can introduce a pipeline of errors. Another promising solution is using cross-lingual transfer learning through pretrained multilingual models. Existing methods train multilingual models with additional code-mixed task data or refine the cross-lingual representations through parallel ontologies. In this work, we enhance the transfer learning process by intermediate fine-tuning of pretrained multilingual models, where the multilingual models are fine-tuned with different but related data and/or tasks. Specifically, we use parallel and conversational movie subtitles datasets to design cross-lingual intermediate tasks suitable for downstream dialogue tasks. We use only 200K lines of parallel data for intermediate fine-tuning which is already available for 1782 language pairs. We test our approach on the cross-lingual dialogue state tracking task for the parallel MultiWoZ (English -\textgreater Chinese, Chinese -\textgreater English) and Multilingual WoZ (English -\textgreater German, English -\textgreater Italian) datasets. We achieve impressive improvements (\textgreater 20% on joint goal accuracy) on the parallel MultiWoZ dataset and the Multilingual WoZ dataset over the vanilla baseline with only 10% of the target language task data and zero-shot setup respectively.
%R 10.18653/v1/2021.emnlp-main.87
%U https://aclanthology.org/2021.emnlp-main.87
%U https://doi.org/10.18653/v1/2021.emnlp-main.87
%P 1137-1150

Markdown (Informal)

[Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking](https://aclanthology.org/2021.emnlp-main.87) (Moghe et al., EMNLP 2021)

• Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking (Moghe et al., EMNLP 2021)

ACL

• Nikita Moghe, Mark Steedman, and Alexandra Birch. 2021. Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1137–1150, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.