<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="holmstrom-doostmohammadi-2023-making">
    <titleInfo>
        <title>Making Instruction Finetuning Accessible to Non-English Languages: A Case Study on Swedish Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Oskar</namePart>
        <namePart type="family">Holmström</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ehsan</namePart>
        <namePart type="family">Doostmohammadi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Tanel</namePart>
            <namePart type="family">Alumäe</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mark</namePart>
            <namePart type="family">Fishel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>University of Tartu Library</publisher>
            <place>
                <placeTerm type="text">Tórshavn, Faroe Islands</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In recent years, instruction finetuning models have received increased attention due to their remarkable zero-shot and generalization capabilities. However, the widespread implementation of these models has been limited to the English language, largely due to the costs and challenges associated with creating instruction datasets. To overcome this, automatic instruction generation has been proposed as a resourceful alternative. We see this as an opportunity for the adoption of instruction finetuning for other languages. In this paper we explore the viability of instruction finetuning for Swedish. We translate a dataset of generated instructions from English to Swedish, using it to finetune both Swedish and non-Swedish models. Results indicate that the use of translated instructions significantly improves the models’ zero-shot performance, even on unseen data, while staying competitive with strong baselines ten times in size. We see this paper is a first step and a proof of concept that instruction finetuning for Swedish is within reach, through resourceful means, and that there exist several directions for further improvements.</abstract>
    <identifier type="citekey">holmstrom-doostmohammadi-2023-making</identifier>
    <location>
        <url>https://aclanthology.org/2023.nodalida-1.62</url>
    </location>
    <part>
        <date>2023-05</date>
        <extent unit="page">
            <start>634</start>
            <end>642</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Making Instruction Finetuning Accessible to Non-English Languages: A Case Study on Swedish Models
%A Holmström, Oskar
%A Doostmohammadi, Ehsan
%Y Alumäe, Tanel
%Y Fishel, Mark
%S Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)
%D 2023
%8 May
%I University of Tartu Library
%C Tórshavn, Faroe Islands
%F holmstrom-doostmohammadi-2023-making
%X In recent years, instruction finetuning models have received increased attention due to their remarkable zero-shot and generalization capabilities. However, the widespread implementation of these models has been limited to the English language, largely due to the costs and challenges associated with creating instruction datasets. To overcome this, automatic instruction generation has been proposed as a resourceful alternative. We see this as an opportunity for the adoption of instruction finetuning for other languages. In this paper we explore the viability of instruction finetuning for Swedish. We translate a dataset of generated instructions from English to Swedish, using it to finetune both Swedish and non-Swedish models. Results indicate that the use of translated instructions significantly improves the models’ zero-shot performance, even on unseen data, while staying competitive with strong baselines ten times in size. We see this paper is a first step and a proof of concept that instruction finetuning for Swedish is within reach, through resourceful means, and that there exist several directions for further improvements.
%U https://aclanthology.org/2023.nodalida-1.62
%P 634-642

Markdown (Informal)

[Making Instruction Finetuning Accessible to Non-English Languages: A Case Study on Swedish Models](https://aclanthology.org/2023.nodalida-1.62) (Holmström & Doostmohammadi, NoDaLiDa 2023)

• Making Instruction Finetuning Accessible to Non-English Languages: A Case Study on Swedish Models (Holmström & Doostmohammadi, NoDaLiDa 2023)

ACL

• Oskar Holmström and Ehsan Doostmohammadi. 2023. Making Instruction Finetuning Accessible to Non-English Languages: A Case Study on Swedish Models. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 634–642, Tórshavn, Faroe Islands. University of Tartu Library.