GroningenAnnotatesGaza at the FIGNEWS 2024 Shared Task: Analyzing Bias in Conflict Narratives
Khalid Khatib, Sara Gemelli, Saskia Heisterborg, Pritha Majumdar, Gosse Minnema, Arianna Muti, Noa Solissa
Abstract
In this paper we report the development of our annotation methodology for the shared task FIGNEWS 2024. The objective of the shared task is to look into the layers of bias in how the war on Gaza is represented in media narrative. Our methodology follows the prescriptive paradigm, in which guidelines are detailed and refined through an iterative process in which edge cases are discussed and converged. Our IAA score (Krippendorff’s 𝛼) is 0.420, highlighting the challenging and subjective nature of the task. Our results show that 52% of posts were unbiased, 42% biased against Palestine, 5% biased against Israel, and 3% biased against both. 16% were unclear or not applicable.- Anthology ID:
- 2024.arabicnlp-1.68
- Volume:
- Proceedings of The Second Arabic Natural Language Processing Conference
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Nizar Habash, Houda Bouamor, Ramy Eskander, Nadi Tomeh, Ibrahim Abu Farha, Ahmed Abdelali, Samia Touileb, Injy Hamed, Yaser Onaizan, Bashar Alhafni, Wissam Antoun, Salam Khalifa, Hatem Haddad, Imed Zitouni, Badr AlKhamissi, Rawan Almatham, Khalil Mrini
- Venues:
- ArabicNLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 630–639
- Language:
- URL:
- https://aclanthology.org/2024.arabicnlp-1.68
- DOI:
- Bibkey:
- Cite (ACL):
- Khalid Khatib, Sara Gemelli, Saskia Heisterborg, Pritha Majumdar, Gosse Minnema, Arianna Muti, and Noa Solissa. 2024. GroningenAnnotatesGaza at the FIGNEWS 2024 Shared Task: Analyzing Bias in Conflict Narratives. In Proceedings of The Second Arabic Natural Language Processing Conference, pages 630–639, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- GroningenAnnotatesGaza at the FIGNEWS 2024 Shared Task: Analyzing Bias in Conflict Narratives (Khatib et al., ArabicNLP-WS 2024)
- Copy Citation:
- PDF:
- https://aclanthology.org/2024.arabicnlp-1.68.pdf
Export citation
@inproceedings{khatib-etal-2024-groningenannotatesgaza, title = "{G}roningen{A}nnotates{G}aza at the {FIGNEWS} 2024 Shared Task: Analyzing Bias in Conflict Narratives", author = "Khatib, Khalid and Gemelli, Sara and Heisterborg, Saskia and Majumdar, Pritha and Minnema, Gosse and Muti, Arianna and Solissa, Noa", editor = "Habash, Nizar and Bouamor, Houda and Eskander, Ramy and Tomeh, Nadi and Abu Farha, Ibrahim and Abdelali, Ahmed and Touileb, Samia and Hamed, Injy and Onaizan, Yaser and Alhafni, Bashar and Antoun, Wissam and Khalifa, Salam and Haddad, Hatem and Zitouni, Imed and AlKhamissi, Badr and Almatham, Rawan and Mrini, Khalil", booktitle = "Proceedings of The Second Arabic Natural Language Processing Conference", month = aug, year = "2024", address = "Bangkok, Thailand", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2024.arabicnlp-1.68", pages = "630--639", abstract = "In this paper we report the development of our annotation methodology for the shared task FIGNEWS 2024. The objective of the shared task is to look into the layers of bias in how the war on Gaza is represented in media narrative. Our methodology follows the prescriptive paradigm, in which guidelines are detailed and refined through an iterative process in which edge cases are discussed and converged. Our IAA score (Krippendorff{'}s $\alpha$) is 0.420, highlighting the challenging and subjective nature of the task. Our results show that 52{\%} of posts were unbiased, 42{\%} biased against Palestine, 5{\%} biased against Israel, and 3{\%} biased against both. 16{\%} were unclear or not applicable.", }
<?xml version="1.0" encoding="UTF-8"?> <modsCollection xmlns="http://www.loc.gov/mods/v3"> <mods ID="khatib-etal-2024-groningenannotatesgaza"> <titleInfo> <title>GroningenAnnotatesGaza at the FIGNEWS 2024 Shared Task: Analyzing Bias in Conflict Narratives</title> </titleInfo> <name type="personal"> <namePart type="given">Khalid</namePart> <namePart type="family">Khatib</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Sara</namePart> <namePart type="family">Gemelli</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Saskia</namePart> <namePart type="family">Heisterborg</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Pritha</namePart> <namePart type="family">Majumdar</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Gosse</namePart> <namePart type="family">Minnema</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Arianna</namePart> <namePart type="family">Muti</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Noa</namePart> <namePart type="family">Solissa</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <originInfo> <dateIssued>2024-08</dateIssued> </originInfo> <typeOfResource>text</typeOfResource> <relatedItem type="host"> <titleInfo> <title>Proceedings of The Second Arabic Natural Language Processing Conference</title> </titleInfo> <name type="personal"> <namePart type="given">Nizar</namePart> <namePart type="family">Habash</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Houda</namePart> <namePart type="family">Bouamor</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Ramy</namePart> <namePart type="family">Eskander</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Nadi</namePart> <namePart type="family">Tomeh</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Ibrahim</namePart> <namePart type="family">Abu Farha</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Ahmed</namePart> <namePart type="family">Abdelali</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Samia</namePart> <namePart type="family">Touileb</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Injy</namePart> <namePart type="family">Hamed</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Yaser</namePart> <namePart type="family">Onaizan</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Bashar</namePart> <namePart type="family">Alhafni</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Wissam</namePart> <namePart type="family">Antoun</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Salam</namePart> <namePart type="family">Khalifa</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Hatem</namePart> <namePart type="family">Haddad</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Imed</namePart> <namePart type="family">Zitouni</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Badr</namePart> <namePart type="family">AlKhamissi</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Rawan</namePart> <namePart type="family">Almatham</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Khalil</namePart> <namePart type="family">Mrini</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <originInfo> <publisher>Association for Computational Linguistics</publisher> <place> <placeTerm type="text">Bangkok, Thailand</placeTerm> </place> </originInfo> <genre authority="marcgt">conference publication</genre> </relatedItem> <abstract>In this paper we report the development of our annotation methodology for the shared task FIGNEWS 2024. The objective of the shared task is to look into the layers of bias in how the war on Gaza is represented in media narrative. Our methodology follows the prescriptive paradigm, in which guidelines are detailed and refined through an iterative process in which edge cases are discussed and converged. Our IAA score (Krippendorff’s α) is 0.420, highlighting the challenging and subjective nature of the task. Our results show that 52% of posts were unbiased, 42% biased against Palestine, 5% biased against Israel, and 3% biased against both. 16% were unclear or not applicable.</abstract> <identifier type="citekey">khatib-etal-2024-groningenannotatesgaza</identifier> <location> <url>https://aclanthology.org/2024.arabicnlp-1.68</url> </location> <part> <date>2024-08</date> <extent unit="page"> <start>630</start> <end>639</end> </extent> </part> </mods> </modsCollection>
%0 Conference Proceedings %T GroningenAnnotatesGaza at the FIGNEWS 2024 Shared Task: Analyzing Bias in Conflict Narratives %A Khatib, Khalid %A Gemelli, Sara %A Heisterborg, Saskia %A Majumdar, Pritha %A Minnema, Gosse %A Muti, Arianna %A Solissa, Noa %Y Habash, Nizar %Y Bouamor, Houda %Y Eskander, Ramy %Y Tomeh, Nadi %Y Abu Farha, Ibrahim %Y Abdelali, Ahmed %Y Touileb, Samia %Y Hamed, Injy %Y Onaizan, Yaser %Y Alhafni, Bashar %Y Antoun, Wissam %Y Khalifa, Salam %Y Haddad, Hatem %Y Zitouni, Imed %Y AlKhamissi, Badr %Y Almatham, Rawan %Y Mrini, Khalil %S Proceedings of The Second Arabic Natural Language Processing Conference %D 2024 %8 August %I Association for Computational Linguistics %C Bangkok, Thailand %F khatib-etal-2024-groningenannotatesgaza %X In this paper we report the development of our annotation methodology for the shared task FIGNEWS 2024. The objective of the shared task is to look into the layers of bias in how the war on Gaza is represented in media narrative. Our methodology follows the prescriptive paradigm, in which guidelines are detailed and refined through an iterative process in which edge cases are discussed and converged. Our IAA score (Krippendorff’s α) is 0.420, highlighting the challenging and subjective nature of the task. Our results show that 52% of posts were unbiased, 42% biased against Palestine, 5% biased against Israel, and 3% biased against both. 16% were unclear or not applicable. %U https://aclanthology.org/2024.arabicnlp-1.68 %P 630-639
Markdown (Informal)
[GroningenAnnotatesGaza at the FIGNEWS 2024 Shared Task: Analyzing Bias in Conflict Narratives](https://aclanthology.org/2024.arabicnlp-1.68) (Khatib et al., ArabicNLP-WS 2024)
- GroningenAnnotatesGaza at the FIGNEWS 2024 Shared Task: Analyzing Bias in Conflict Narratives (Khatib et al., ArabicNLP-WS 2024)
ACL
- Khalid Khatib, Sara Gemelli, Saskia Heisterborg, Pritha Majumdar, Gosse Minnema, Arianna Muti, and Noa Solissa. 2024. GroningenAnnotatesGaza at the FIGNEWS 2024 Shared Task: Analyzing Bias in Conflict Narratives. In Proceedings of The Second Arabic Natural Language Processing Conference, pages 630–639, Bangkok, Thailand. Association for Computational Linguistics.