<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="das-etal-2022-hatecheckhin">
    <titleInfo>
        <title>HateCheckHIn: Evaluating Hindi Hate Speech Detection Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Mithun</namePart>
        <namePart type="family">Das</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Punyajoy</namePart>
        <namePart type="family">Saha</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Binny</namePart>
        <namePart type="family">Mathew</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Animesh</namePart>
        <namePart type="family">Mukherjee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Thirteenth Language Resources and Evaluation Conference</title>
        </titleInfo>
        <originInfo>
            <publisher>European Language Resources Association</publisher>
            <place>
                <placeTerm type="text">Marseille, France</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Due to the sheer volume of online hate, the AI and NLP communities have started building models to detect such hateful content. Recently, multilingual hate is a major emerging challenge for automated detection where code-mixing or more than one language have been used for conversation in social media. Typically, hate speech detection models are evaluated by measuring their performance on the held-out test data using metrics such as accuracy and F1-score. While these metrics are useful, it becomes difficult to identify using them where the model is failing, and how to resolve it. To enable more targeted diagnostic insights of such multilingual hate speech models, we introduce a set of functionalities for the purpose of evaluation. We have been inspired to design this kind of functionalities based on real-world conversation on social media. Considering Hindi as a base language, we craft test cases for each functionality. We name our evaluation dataset HateCheckHIn. To illustrate the utility of these functionalities , we test state-of-the-art transformer based m-BERT model and the Perspective API.</abstract>
    <identifier type="citekey">das-etal-2022-hatecheckhin</identifier>
    <location>
        <url>https://aclanthology.org/2022.lrec-1.575</url>
    </location>
    <part>
        <date>2022-06</date>
        <extent unit="page">
            <start>5378</start>
            <end>5387</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T HateCheckHIn: Evaluating Hindi Hate Speech Detection Models
%A Das, Mithun
%A Saha, Punyajoy
%A Mathew, Binny
%A Mukherjee, Animesh
%S Proceedings of the Thirteenth Language Resources and Evaluation Conference
%D 2022
%8 June
%I European Language Resources Association
%C Marseille, France
%F das-etal-2022-hatecheckhin
%X Due to the sheer volume of online hate, the AI and NLP communities have started building models to detect such hateful content. Recently, multilingual hate is a major emerging challenge for automated detection where code-mixing or more than one language have been used for conversation in social media. Typically, hate speech detection models are evaluated by measuring their performance on the held-out test data using metrics such as accuracy and F1-score. While these metrics are useful, it becomes difficult to identify using them where the model is failing, and how to resolve it. To enable more targeted diagnostic insights of such multilingual hate speech models, we introduce a set of functionalities for the purpose of evaluation. We have been inspired to design this kind of functionalities based on real-world conversation on social media. Considering Hindi as a base language, we craft test cases for each functionality. We name our evaluation dataset HateCheckHIn. To illustrate the utility of these functionalities , we test state-of-the-art transformer based m-BERT model and the Perspective API.
%U https://aclanthology.org/2022.lrec-1.575
%P 5378-5387

Markdown (Informal)

[HateCheckHIn: Evaluating Hindi Hate Speech Detection Models](https://aclanthology.org/2022.lrec-1.575) (Das et al., LREC 2022)

• HateCheckHIn: Evaluating Hindi Hate Speech Detection Models (Das et al., LREC 2022)

ACL

• Mithun Das, Punyajoy Saha, Binny Mathew, and Animesh Mukherjee. 2022. HateCheckHIn: Evaluating Hindi Hate Speech Detection Models. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5378–5387, Marseille, France. European Language Resources Association.