@inproceedings{abubakar-etal-2026-well,
title = "How Well Does It Generalize? A Robustness Study of {H}ausa Machine-Generated Text Detection",
author = "Abubakar, Abdulhamid and
Adam, Faisal Muhammad and
Muhammad, Shamsuddeen Hassan and
Khalid, Abubakar Auwal and
Sani, Babangida",
editor = "Zhang, Chen and
Allaway, Emily and
M{'}hamdi, Meryem and
Limkonchotiwat, Peerat and
Bai, Richard He and
T.Y.S.S., Santosh and
Thapa, Surendrabikram and
Li, Yinqiao",
booktitle = "Proceedings of the 10th Widening {NLP} Workshop",
month = oct,
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.winlp-main.5/",
pages = "48--60",
ISBN = "979-8-89176-472-9",
abstract = "Sani et al. (2025) introduced the first dedicated Hausa machine-generated text (MGT) detec- tor, an AfroXLMR-large classifier trained on 2,586 paired articles from seven Hausa news outlets and Gemini 2.0 Flash, reporting F1 of 0.9921 on a headline-grouped split. They iden- tified three limitations of their work: a single news domain, a small batch and short train- ing schedule, and a single generator (Gemini 2.0 Flash) for the machine-generated data. We extend their work along the third axis. First, we replicate their reported result across four random seeds, obtaining F1 of 0.992 {\ensuremath{\pm}}0.005. Test recall on the AI class is 1.000 at every seed. Second, we analyse the detector{'}s false positives. All errors fall on Human articles, and two native-speaker reviewers confirm these are genuine, natural Hausa journalism. The clear- est correlate of misclassification is the absence of numerical content: the median digit frac- tion is 0.0{\%} in misclassified articles vs. 0.6{\%} in correctly classified ones. Third, we test the detector on six non-training generators span- ning four model families. Detection rates span from 0{\%} (Aya-Expanse 8B) to nearly 100{\%} (Gemma 4). All three Google-family models in our experiment (training-distribution Gemini 2.0 Flash and the two non-training Gemma 4 variants) are detected at rates above 98{\%} with mean P(machine) above 0.98. No non-Google generator exceeds 21{\%}. Three contamination checks fail to explain this away: generated articles overlap very little with their human sources, detection barely moves when we re- move every article that shares a 13-gram with its source, and the Google output that gets de- tected is at least as diverse as everyone else{'}s, so it is not memorised or templated text. What the detector has learned looks less like a general Hausa MGT signal and more like a Gemini- family stylistic fingerprint. We discuss what this means for deployment, including a 3.1{\%} false-positive rate on real Hausa journalists."
}<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="abubakar-etal-2026-well">
<titleInfo>
<title>How Well Does It Generalize? A Robustness Study of Hausa Machine-Generated Text Detection</title>
</titleInfo>
<name type="personal">
<namePart type="given">Abdulhamid</namePart>
<namePart type="family">Abubakar</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Faisal</namePart>
<namePart type="given">Muhammad</namePart>
<namePart type="family">Adam</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Shamsuddeen</namePart>
<namePart type="given">Hassan</namePart>
<namePart type="family">Muhammad</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Abubakar</namePart>
<namePart type="given">Auwal</namePart>
<namePart type="family">Khalid</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Babangida</namePart>
<namePart type="family">Sani</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<originInfo>
<dateIssued>2026-10</dateIssued>
</originInfo>
<typeOfResource>text</typeOfResource>
<relatedItem type="host">
<titleInfo>
<title>Proceedings of the 10th Widening NLP Workshop</title>
</titleInfo>
<name type="personal">
<namePart type="given">Chen</namePart>
<namePart type="family">Zhang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Emily</namePart>
<namePart type="family">Allaway</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Meryem</namePart>
<namePart type="family">M’hamdi</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Peerat</namePart>
<namePart type="family">Limkonchotiwat</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Richard</namePart>
<namePart type="given">He</namePart>
<namePart type="family">Bai</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Santosh</namePart>
<namePart type="family">T.Y.S.S.</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Surendrabikram</namePart>
<namePart type="family">Thapa</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yinqiao</namePart>
<namePart type="family">Li</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<originInfo>
<publisher>Association for Computational Linguistics</publisher>
<place>
<placeTerm type="text">Budapest, Hungary</placeTerm>
</place>
</originInfo>
<genre authority="marcgt">conference publication</genre>
<identifier type="isbn">979-8-89176-472-9</identifier>
</relatedItem>
<abstract>Sani et al. (2025) introduced the first dedicated Hausa machine-generated text (MGT) detec- tor, an AfroXLMR-large classifier trained on 2,586 paired articles from seven Hausa news outlets and Gemini 2.0 Flash, reporting F1 of 0.9921 on a headline-grouped split. They iden- tified three limitations of their work: a single news domain, a small batch and short train- ing schedule, and a single generator (Gemini 2.0 Flash) for the machine-generated data. We extend their work along the third axis. First, we replicate their reported result across four random seeds, obtaining F1 of 0.992 \ensuremath\pm0.005. Test recall on the AI class is 1.000 at every seed. Second, we analyse the detector’s false positives. All errors fall on Human articles, and two native-speaker reviewers confirm these are genuine, natural Hausa journalism. The clear- est correlate of misclassification is the absence of numerical content: the median digit frac- tion is 0.0% in misclassified articles vs. 0.6% in correctly classified ones. Third, we test the detector on six non-training generators span- ning four model families. Detection rates span from 0% (Aya-Expanse 8B) to nearly 100% (Gemma 4). All three Google-family models in our experiment (training-distribution Gemini 2.0 Flash and the two non-training Gemma 4 variants) are detected at rates above 98% with mean P(machine) above 0.98. No non-Google generator exceeds 21%. Three contamination checks fail to explain this away: generated articles overlap very little with their human sources, detection barely moves when we re- move every article that shares a 13-gram with its source, and the Google output that gets de- tected is at least as diverse as everyone else’s, so it is not memorised or templated text. What the detector has learned looks less like a general Hausa MGT signal and more like a Gemini- family stylistic fingerprint. We discuss what this means for deployment, including a 3.1% false-positive rate on real Hausa journalists.</abstract>
<identifier type="citekey">abubakar-etal-2026-well</identifier>
<location>
<url>https://aclanthology.org/2026.winlp-main.5/</url>
</location>
<part>
<date>2026-10</date>
<extent unit="page">
<start>48</start>
<end>60</end>
</extent>
</part>
</mods>
</modsCollection>
%0 Conference Proceedings
%T How Well Does It Generalize? A Robustness Study of Hausa Machine-Generated Text Detection
%A Abubakar, Abdulhamid
%A Adam, Faisal Muhammad
%A Muhammad, Shamsuddeen Hassan
%A Khalid, Abubakar Auwal
%A Sani, Babangida
%Y Zhang, Chen
%Y Allaway, Emily
%Y M’hamdi, Meryem
%Y Limkonchotiwat, Peerat
%Y Bai, Richard He
%Y T.Y.S.S., Santosh
%Y Thapa, Surendrabikram
%Y Li, Yinqiao
%S Proceedings of the 10th Widening NLP Workshop
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-472-9
%F abubakar-etal-2026-well
%X Sani et al. (2025) introduced the first dedicated Hausa machine-generated text (MGT) detec- tor, an AfroXLMR-large classifier trained on 2,586 paired articles from seven Hausa news outlets and Gemini 2.0 Flash, reporting F1 of 0.9921 on a headline-grouped split. They iden- tified three limitations of their work: a single news domain, a small batch and short train- ing schedule, and a single generator (Gemini 2.0 Flash) for the machine-generated data. We extend their work along the third axis. First, we replicate their reported result across four random seeds, obtaining F1 of 0.992 \ensuremath\pm0.005. Test recall on the AI class is 1.000 at every seed. Second, we analyse the detector’s false positives. All errors fall on Human articles, and two native-speaker reviewers confirm these are genuine, natural Hausa journalism. The clear- est correlate of misclassification is the absence of numerical content: the median digit frac- tion is 0.0% in misclassified articles vs. 0.6% in correctly classified ones. Third, we test the detector on six non-training generators span- ning four model families. Detection rates span from 0% (Aya-Expanse 8B) to nearly 100% (Gemma 4). All three Google-family models in our experiment (training-distribution Gemini 2.0 Flash and the two non-training Gemma 4 variants) are detected at rates above 98% with mean P(machine) above 0.98. No non-Google generator exceeds 21%. Three contamination checks fail to explain this away: generated articles overlap very little with their human sources, detection barely moves when we re- move every article that shares a 13-gram with its source, and the Google output that gets de- tected is at least as diverse as everyone else’s, so it is not memorised or templated text. What the detector has learned looks less like a general Hausa MGT signal and more like a Gemini- family stylistic fingerprint. We discuss what this means for deployment, including a 3.1% false-positive rate on real Hausa journalists.
%U https://aclanthology.org/2026.winlp-main.5/
%P 48-60
Markdown (Informal)
[How Well Does It Generalize? A Robustness Study of Hausa Machine-Generated Text Detection](https://aclanthology.org/2026.winlp-main.5/) (Abubakar et al., WiNLP 2026)
ACL