<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lo-wu-2013-human">
    <titleInfo>
        <title>Human semantic MT evaluation with HMEANT for IWSLT 2013</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Chi-kiu</namePart>
        <namePart type="family">Lo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dekai</namePart>
        <namePart type="family">Wu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2013-dec 5-6</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 10th International Workshop on Spoken Language Translation: Evaluation Campaign</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Joy</namePart>
            <namePart type="given">Ying</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <place>
                <placeTerm type="text">Heidelberg, Germany</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We present the results of large-scale human semantic MT evaluation with HMEANT on the IWSLT 2013 German-English MT and SLT tracks and show that HMEANT evaluates the performance of the MT systems differently compared to BLEU and TER. Together with the references, all the translations are annotated by annotators who are native English speakers in both semantic role labeling stage and role filler alignment stage of HMEANT. We obtain high inter-annotator agreement and low annotation time costs which indicate that it is feasible to run a large-scale human semantic MT evaluation campaign using HMEANT. Our results also show that HMEANT is a robust and reliable semantic MT evaluation metric for running large-scale evaluation campaigns as it is inexpensive and simple while maintaining the semantic representational transparency to provide a perspective which is different from BLEU and TER in order to understand the performance of the state-of-the-art MT systems.</abstract>
    <identifier type="citekey">lo-wu-2013-human</identifier>
    <location>
        <url>https://aclanthology.org/2013.iwslt-evaluation.2</url>
    </location>
    <part>
        <date>2013-dec 5-6</date>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Human semantic MT evaluation with HMEANT for IWSLT 2013
%A Lo, Chi-kiu
%A Wu, Dekai
%Y Zhang, Joy Ying
%S Proceedings of the 10th International Workshop on Spoken Language Translation: Evaluation Campaign
%D 2013
%8 dec 5 6
%C Heidelberg, Germany
%F lo-wu-2013-human
%X We present the results of large-scale human semantic MT evaluation with HMEANT on the IWSLT 2013 German-English MT and SLT tracks and show that HMEANT evaluates the performance of the MT systems differently compared to BLEU and TER. Together with the references, all the translations are annotated by annotators who are native English speakers in both semantic role labeling stage and role filler alignment stage of HMEANT. We obtain high inter-annotator agreement and low annotation time costs which indicate that it is feasible to run a large-scale human semantic MT evaluation campaign using HMEANT. Our results also show that HMEANT is a robust and reliable semantic MT evaluation metric for running large-scale evaluation campaigns as it is inexpensive and simple while maintaining the semantic representational transparency to provide a perspective which is different from BLEU and TER in order to understand the performance of the state-of-the-art MT systems.
%U https://aclanthology.org/2013.iwslt-evaluation.2

Markdown (Informal)

[Human semantic MT evaluation with HMEANT for IWSLT 2013](https://aclanthology.org/2013.iwslt-evaluation.2) (Lo & Wu, IWSLT 2013)

• Human semantic MT evaluation with HMEANT for IWSLT 2013 (Lo & Wu, IWSLT 2013)

ACL

• Chi-kiu Lo and Dekai Wu. 2013. Human semantic MT evaluation with HMEANT for IWSLT 2013. In Proceedings of the 10th International Workshop on Spoken Language Translation: Evaluation Campaign, Heidelberg, Germany.