<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="park-caragea-2023-multi">
    <titleInfo>
        <title>Multi-Task Knowledge Distillation with Embedding Constraints for Scholarly Keyphrase Boundary Classification</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Seo</namePart>
        <namePart type="family">Park</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Cornelia</namePart>
        <namePart type="family">Caragea</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The task of scholarly keyphrase boundary classification aims at identifying keyphrases from scientific papers and classifying them with their types from a set of predefined classes (e.g., task, process, or material). Despite the importance of keyphrases and their types in many downstream applications including indexing, searching, and question answering over scientific documents, scholarly keyphrase boundary classification is still an under-explored task. In this work, we propose a novel embedding constraint on multi-task knowledge distillation which enforces the teachers (single-task models) and the student (multi-task model) similarity in the embedding space. Specifically, we enforce that the student model is trained not only to imitate the teachers’ output distribution over classes, but also to produce language representations that are similar to those produced by the teachers. Our results show that the proposed approach outperforms previous works and strong baselines on three datasets of scientific documents.</abstract>
    <identifier type="citekey">park-caragea-2023-multi</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.805</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.805</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>13026</start>
            <end>13042</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Multi-Task Knowledge Distillation with Embedding Constraints for Scholarly Keyphrase Boundary Classification
%A Park, Seo
%A Caragea, Cornelia
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F park-caragea-2023-multi
%X The task of scholarly keyphrase boundary classification aims at identifying keyphrases from scientific papers and classifying them with their types from a set of predefined classes (e.g., task, process, or material). Despite the importance of keyphrases and their types in many downstream applications including indexing, searching, and question answering over scientific documents, scholarly keyphrase boundary classification is still an under-explored task. In this work, we propose a novel embedding constraint on multi-task knowledge distillation which enforces the teachers (single-task models) and the student (multi-task model) similarity in the embedding space. Specifically, we enforce that the student model is trained not only to imitate the teachers’ output distribution over classes, but also to produce language representations that are similar to those produced by the teachers. Our results show that the proposed approach outperforms previous works and strong baselines on three datasets of scientific documents.
%R 10.18653/v1/2023.emnlp-main.805
%U https://aclanthology.org/2023.emnlp-main.805
%U https://doi.org/10.18653/v1/2023.emnlp-main.805
%P 13026-13042

Markdown (Informal)

[Multi-Task Knowledge Distillation with Embedding Constraints for Scholarly Keyphrase Boundary Classification](https://aclanthology.org/2023.emnlp-main.805) (Park & Caragea, EMNLP 2023)

• Multi-Task Knowledge Distillation with Embedding Constraints for Scholarly Keyphrase Boundary Classification (Park & Caragea, EMNLP 2023)

ACL

• Seo Park and Cornelia Caragea. 2023. Multi-Task Knowledge Distillation with Embedding Constraints for Scholarly Keyphrase Boundary Classification. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13026–13042, Singapore. Association for Computational Linguistics.