<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="mewes-etal-2026-gruff">
    <titleInfo>
        <title>GRUFF: LLM Pronoun Fidelity, Reasoning, and Biases in German</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Fabian</namePart>
        <namePart type="family">Mewes</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anne</namePart>
        <namePart type="family">Lauscher</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vagrant</namePart>
        <namePart type="family">Gautam</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 7th Workshop on Gender Bias in Natural Language Processing (GeBNLP)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Agnieszka</namePart>
            <namePart type="family">Faleńska</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Christine</namePart>
            <namePart type="family">Basta</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marta</namePart>
            <namePart type="family">Costa-jussà</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Karolina</namePart>
            <namePart type="family">Stańczak</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Debora</namePart>
            <namePart type="family">Nozza</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Giuseppe</namePart>
            <namePart type="family">Attanasio</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vera</namePart>
            <namePart type="family">Neplenbroek</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Christian</namePart>
            <namePart type="family">Hardmeier</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hengqin, China (remote)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-447-7</identifier>
    </relatedItem>
    <abstract>Third-person singular pronouns have long been used to study stereotypical biases in language models and to test their abilities to reason about reference. More recently, the interplay between reasoning and bias has been investigated with the task of pronoun fidelity, which assesses models’ abilities to correctly reuse a previously-specified pronoun for a discourse entity, independent of other potentially distracting discourse entities mentioned in between. However, such research focuses on English, which is a language with limited grammatical gender and almost no gender agreement. In this paper we contribute a novel, large-scale dataset, **GRUFF**, to measure pronoun fidelity in German, covering four different gender agreement systems in nouns, and four sets of pronouns. With this dataset, we show that LLMs display strong grammatical agreement for masculine and feminine entities in the absence of explicit context, but not for neopronouns _xier_ and _en_. In the presence of distractors, however, models degrade on all pronoun sets. Finally, we show that occupational stereotypes in this context are poorly correlated across grammatical cases, and across most models, except ones with closely related architectures. We release all code and data to encourage further work on gender-inclusive language and referential reasoning in German.</abstract>
    <identifier type="citekey">mewes-etal-2026-gruff</identifier>
    <location>
        <url>https://aclanthology.org/2026.gebnlp-1.4/</url>
    </location>
    <part>
        <date>2026-11</date>
        <extent unit="page">
            <start>35</start>
            <end>51</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T GRUFF: LLM Pronoun Fidelity, Reasoning, and Biases in German
%A Mewes, Fabian
%A Lauscher, Anne
%A Gautam, Vagrant
%Y Faleńska, Agnieszka
%Y Basta, Christine
%Y Costa-jussà, Marta
%Y Stańczak, Karolina
%Y Nozza, Debora
%Y Attanasio, Giuseppe
%Y Neplenbroek, Vera
%Y Hardmeier, Christian
%S Proceedings of the 7th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
%D 2026
%8 November
%I Association for Computational Linguistics
%C Hengqin, China (remote)
%@ 979-8-89176-447-7
%F mewes-etal-2026-gruff
%X Third-person singular pronouns have long been used to study stereotypical biases in language models and to test their abilities to reason about reference. More recently, the interplay between reasoning and bias has been investigated with the task of pronoun fidelity, which assesses models’ abilities to correctly reuse a previously-specified pronoun for a discourse entity, independent of other potentially distracting discourse entities mentioned in between. However, such research focuses on English, which is a language with limited grammatical gender and almost no gender agreement. In this paper we contribute a novel, large-scale dataset, **GRUFF**, to measure pronoun fidelity in German, covering four different gender agreement systems in nouns, and four sets of pronouns. With this dataset, we show that LLMs display strong grammatical agreement for masculine and feminine entities in the absence of explicit context, but not for neopronouns _xier_ and _en_. In the presence of distractors, however, models degrade on all pronoun sets. Finally, we show that occupational stereotypes in this context are poorly correlated across grammatical cases, and across most models, except ones with closely related architectures. We release all code and data to encourage further work on gender-inclusive language and referential reasoning in German.
%U https://aclanthology.org/2026.gebnlp-1.4/
%P 35-51

Markdown (Informal)

[GRUFF: LLM Pronoun Fidelity, Reasoning, and Biases in German](https://aclanthology.org/2026.gebnlp-1.4/) (Mewes et al., GeBNLP 2026)

• GRUFF: LLM Pronoun Fidelity, Reasoning, and Biases in German (Mewes et al., GeBNLP 2026)

ACL

• Fabian Mewes, Anne Lauscher, and Vagrant Gautam. 2026. GRUFF: LLM Pronoun Fidelity, Reasoning, and Biases in German. In Proceedings of the 7th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 35–51, Hengqin, China (remote). Association for Computational Linguistics.