<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="salman-etal-2024-tiny">
    <titleInfo>
        <title>Tiny But Mighty: A Crowdsourced Benchmark Dataset for Triple Extraction from Unstructured Text</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Muhammad</namePart>
        <namePart type="family">Salman</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Armin</namePart>
        <namePart type="family">Haller</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sergio</namePart>
        <namePart type="given">J</namePart>
        <namePart type="family">Rodriguez Mendez</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Usman</namePart>
        <namePart type="family">Naseem</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 20th Joint ACL - ISO Workshop on Interoperable Semantic Annotation @ LREC-COLING 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Harry</namePart>
            <namePart type="family">Bunt</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nancy</namePart>
            <namePart type="family">Ide</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kiyong</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Volha</namePart>
            <namePart type="family">Petukhova</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">James</namePart>
            <namePart type="family">Pustejovsky</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Laurent</namePart>
            <namePart type="family">Romary</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA and ICCL</publisher>
            <place>
                <placeTerm type="text">Torino, Italia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In the context of Natural Language Processing (NLP) and Semantic Web applications, constructing Knowledge Graphs (KGs) from unstructured text plays a vital role. Several techniques have been developed for KG construction from text, but the lack of standardized datasets hinders the evaluation of triple extraction methods. The evaluation of existing KG construction approaches is based on structured data or manual investigations. To overcome this limitation, this work introduces a novel dataset specifically designed to evaluate KG construction techniques from unstructured text. Our dataset consists of a diverse collection of compound and complex sentences meticulously annotated by human annotators with potential triples (subject, verb, object). The annotations underwent further scrutiny by expert ontologists to ensure accuracy and consistency. For evaluation purposes, the proposed F-measure criterion offers a robust approach to quantify the relatedness and assess the alignment between extracted triples and the ground-truth triples, providing a valuable tool for evaluating the performance of triple extraction systems. By providing a diverse collection of high-quality triples, our proposed benchmark dataset offers a comprehensive training and evaluation set for refining the performance of state-of-the-art language models on a triple extraction task. Furthermore, this dataset encompasses various KG-related tasks, such as named entity recognition, relation extraction, and entity linking.</abstract>
    <identifier type="citekey">salman-etal-2024-tiny</identifier>
    <location>
        <url>https://aclanthology.org/2024.isa-1.10</url>
    </location>
    <part>
        <date>2024-05</date>
        <extent unit="page">
            <start>71</start>
            <end>81</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Tiny But Mighty: A Crowdsourced Benchmark Dataset for Triple Extraction from Unstructured Text
%A Salman, Muhammad
%A Haller, Armin
%A Rodriguez Mendez, Sergio J.
%A Naseem, Usman
%Y Bunt, Harry
%Y Ide, Nancy
%Y Lee, Kiyong
%Y Petukhova, Volha
%Y Pustejovsky, James
%Y Romary, Laurent
%S Proceedings of the 20th Joint ACL - ISO Workshop on Interoperable Semantic Annotation @ LREC-COLING 2024
%D 2024
%8 May
%I ELRA and ICCL
%C Torino, Italia
%F salman-etal-2024-tiny
%X In the context of Natural Language Processing (NLP) and Semantic Web applications, constructing Knowledge Graphs (KGs) from unstructured text plays a vital role. Several techniques have been developed for KG construction from text, but the lack of standardized datasets hinders the evaluation of triple extraction methods. The evaluation of existing KG construction approaches is based on structured data or manual investigations. To overcome this limitation, this work introduces a novel dataset specifically designed to evaluate KG construction techniques from unstructured text. Our dataset consists of a diverse collection of compound and complex sentences meticulously annotated by human annotators with potential triples (subject, verb, object). The annotations underwent further scrutiny by expert ontologists to ensure accuracy and consistency. For evaluation purposes, the proposed F-measure criterion offers a robust approach to quantify the relatedness and assess the alignment between extracted triples and the ground-truth triples, providing a valuable tool for evaluating the performance of triple extraction systems. By providing a diverse collection of high-quality triples, our proposed benchmark dataset offers a comprehensive training and evaluation set for refining the performance of state-of-the-art language models on a triple extraction task. Furthermore, this dataset encompasses various KG-related tasks, such as named entity recognition, relation extraction, and entity linking.
%U https://aclanthology.org/2024.isa-1.10
%P 71-81

Markdown (Informal)

[Tiny But Mighty: A Crowdsourced Benchmark Dataset for Triple Extraction from Unstructured Text](https://aclanthology.org/2024.isa-1.10) (Salman et al., ISA-WS 2024)

• Tiny But Mighty: A Crowdsourced Benchmark Dataset for Triple Extraction from Unstructured Text (Salman et al., ISA-WS 2024)

ACL

• Muhammad Salman, Armin Haller, Sergio J. Rodriguez Mendez, and Usman Naseem. 2024. Tiny But Mighty: A Crowdsourced Benchmark Dataset for Triple Extraction from Unstructured Text. In Proceedings of the 20th Joint ACL - ISO Workshop on Interoperable Semantic Annotation @ LREC-COLING 2024, pages 71–81, Torino, Italia. ELRA and ICCL.