<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hua-etal-2023-mixed">
    <titleInfo>
        <title>Mixed-domain Language Modeling for Processing Long Legal Documents</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Wenyue</namePart>
        <namePart type="family">Hua</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yuchen</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Zhe</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Josie</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Melanie</namePart>
        <namePart type="family">Weber</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Natural Legal Language Processing Workshop 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Daniel</namePart>
            <namePart type="family">Preoțiuc-Pietro</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Catalina</namePart>
            <namePart type="family">Goanta</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ilias</namePart>
            <namePart type="family">Chalkidis</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Leslie</namePart>
            <namePart type="family">Barrett</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Gerasimos</namePart>
            <namePart type="family">Spanakis</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nikolaos</namePart>
            <namePart type="family">Aletras</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The application of Natural Language Processing (NLP) to specialized domains, such as the law, has recently received a surge of interest. As many legal services rely on processing and analyzing large collections of documents, automating such tasks with NLP tools such as language models emerges as a key challenge since legal documents may contain specialized vocabulary from other domains, such as medical terminology in personal injury text. However, most language models are general-purpose models, which either have limited reasoning capabilities on highly specialized legal terminology and syntax, such as BERT or ROBERTA, or are expensive to run and tune, such as GPT-3.5 and Claude. Thus, in this paper, we propose a specialized language model for personal injury text, LEGALRELECTRA, which is trained on mixed-domain legal and medical corpora. We show that as a small language model, our model improves over general-domain and single-domain medical and legal language models when processing mixed-domain (personal injury) text. Our training architecture implements the ELECTRA framework but utilizes REFORMER instead of BERT for its generator and discriminator. We show that this improves the model‘s performance on processing long passages and results in better long-range text comprehension.</abstract>
    <identifier type="citekey">hua-etal-2023-mixed</identifier>
    <identifier type="doi">10.18653/v1/2023.nllp-1.7</identifier>
    <location>
        <url>https://aclanthology.org/2023.nllp-1.7/</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>51</start>
            <end>61</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Mixed-domain Language Modeling for Processing Long Legal Documents
%A Hua, Wenyue
%A Zhang, Yuchen
%A Chen, Zhe
%A Li, Josie
%A Weber, Melanie
%Y Preoțiuc-Pietro, Daniel
%Y Goanta, Catalina
%Y Chalkidis, Ilias
%Y Barrett, Leslie
%Y Spanakis, Gerasimos
%Y Aletras, Nikolaos
%S Proceedings of the Natural Legal Language Processing Workshop 2023
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F hua-etal-2023-mixed
%X The application of Natural Language Processing (NLP) to specialized domains, such as the law, has recently received a surge of interest. As many legal services rely on processing and analyzing large collections of documents, automating such tasks with NLP tools such as language models emerges as a key challenge since legal documents may contain specialized vocabulary from other domains, such as medical terminology in personal injury text. However, most language models are general-purpose models, which either have limited reasoning capabilities on highly specialized legal terminology and syntax, such as BERT or ROBERTA, or are expensive to run and tune, such as GPT-3.5 and Claude. Thus, in this paper, we propose a specialized language model for personal injury text, LEGALRELECTRA, which is trained on mixed-domain legal and medical corpora. We show that as a small language model, our model improves over general-domain and single-domain medical and legal language models when processing mixed-domain (personal injury) text. Our training architecture implements the ELECTRA framework but utilizes REFORMER instead of BERT for its generator and discriminator. We show that this improves the model‘s performance on processing long passages and results in better long-range text comprehension.
%R 10.18653/v1/2023.nllp-1.7
%U https://aclanthology.org/2023.nllp-1.7/
%U https://doi.org/10.18653/v1/2023.nllp-1.7
%P 51-61

Markdown (Informal)

[Mixed-domain Language Modeling for Processing Long Legal Documents](https://aclanthology.org/2023.nllp-1.7/) (Hua et al., NLLP 2023)

• Mixed-domain Language Modeling for Processing Long Legal Documents (Hua et al., NLLP 2023)

ACL

• Wenyue Hua, Yuchen Zhang, Zhe Chen, Josie Li, and Melanie Weber. 2023. Mixed-domain Language Modeling for Processing Long Legal Documents. In Proceedings of the Natural Legal Language Processing Workshop 2023, pages 51–61, Singapore. Association for Computational Linguistics.