<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="nastase-etal-2024-exploring">
    <titleInfo>
        <title>Exploring Italian Sentence Embeddings Properties through Multi-tasking</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Vivi</namePart>
        <namePart type="family">Nastase</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Giuseppe</namePart>
        <namePart type="family">Samo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chunyang</namePart>
        <namePart type="family">Jiang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Paola</namePart>
        <namePart type="family">Merlo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Felice</namePart>
            <namePart type="family">Dell’Orletta</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alessandro</namePart>
            <namePart type="family">Lenci</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Simonetta</namePart>
            <namePart type="family">Montemagni</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rachele</namePart>
            <namePart type="family">Sprugnoli</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>CEUR Workshop Proceedings</publisher>
            <place>
                <placeTerm type="text">Pisa, Italy</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-12-210-7060-6</identifier>
    </relatedItem>
    <abstract>We investigate to what degree existing LLMs encode abstract linguistic information in Italian in a multi-task setting. We exploit curated synthetic data on a large scale – several Blackbird Language Matrices (BLMs) problems in Italian – and use them to study how sentence representations built using pre-trained language models encode specific syntactic and semantic information. We use a two-level architecture to model separately a compression of the sentence embeddings into a representation that contains relevant information for a task, and a BLM task. We then investigate whether we can obtain compressed sentence representations that encode syntactic and semantic information relevant to several BLM tasks. While we expected that the sentence structure – in terms of sequence of phrases/chunks – and chunk properties could be shared across tasks, performance and error analysis show that the clues for the different tasks are encoded in different manners in the sentence embeddings, suggesting that abstract linguistic notions such as constituents or thematic roles does not seem to be present in the pretrained sentence embeddings.</abstract>
    <identifier type="citekey">nastase-etal-2024-exploring</identifier>
    <location>
        <url>https://aclanthology.org/2024.clicit-1.70/</url>
    </location>
    <part>
        <date>2024-12</date>
        <extent unit="page">
            <start>620</start>
            <end>630</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Exploring Italian Sentence Embeddings Properties through Multi-tasking
%A Nastase, Vivi
%A Samo, Giuseppe
%A Jiang, Chunyang
%A Merlo, Paola
%Y Dell’Orletta, Felice
%Y Lenci, Alessandro
%Y Montemagni, Simonetta
%Y Sprugnoli, Rachele
%S Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024)
%D 2024
%8 December
%I CEUR Workshop Proceedings
%C Pisa, Italy
%@ 979-12-210-7060-6
%F nastase-etal-2024-exploring
%X We investigate to what degree existing LLMs encode abstract linguistic information in Italian in a multi-task setting. We exploit curated synthetic data on a large scale – several Blackbird Language Matrices (BLMs) problems in Italian – and use them to study how sentence representations built using pre-trained language models encode specific syntactic and semantic information. We use a two-level architecture to model separately a compression of the sentence embeddings into a representation that contains relevant information for a task, and a BLM task. We then investigate whether we can obtain compressed sentence representations that encode syntactic and semantic information relevant to several BLM tasks. While we expected that the sentence structure – in terms of sequence of phrases/chunks – and chunk properties could be shared across tasks, performance and error analysis show that the clues for the different tasks are encoded in different manners in the sentence embeddings, suggesting that abstract linguistic notions such as constituents or thematic roles does not seem to be present in the pretrained sentence embeddings.
%U https://aclanthology.org/2024.clicit-1.70/
%P 620-630

Markdown (Informal)

[Exploring Italian Sentence Embeddings Properties through Multi-tasking](https://aclanthology.org/2024.clicit-1.70/) (Nastase et al., CLiC-it 2024)

• Exploring Italian Sentence Embeddings Properties through Multi-tasking (Nastase et al., CLiC-it 2024)

ACL

• Vivi Nastase, Giuseppe Samo, Chunyang Jiang, and Paola Merlo. 2024. Exploring Italian Sentence Embeddings Properties through Multi-tasking. In Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), pages 620–630, Pisa, Italy. CEUR Workshop Proceedings.