Investigations of gender-neutral and non-binary language in Romance languages often fail to take into account learner language. There is a lack of material to study how grammatical error correction models handle non-binary language which uses new forms or morphemes. In this paper, we offer Valicə a gender-neutralized version of the Valico treebank, containing Italian learner texts and their corrections. We present the procedure to gender-neutralize Italian sentences, both grammatically correct and incorrect. We implement two different systems to identify which tokens should be neutralized: a rule-based system and a machine learning system, both of which prove equally efficient. The token swap is rule-based, using the neomorphemes -ə and -ɜ to signal singular and plural gender-neutrality. The Valicə dataset can be used to evaluate or train language learning applications for learners of Italian.

Reporting bias is the tendency for speakers to omit unnecessary or obvious information while mentioning things they consider relevant or surprising. In descriptions of people, reporting bias can manifest as a tendency to over report on attributes that deviate from the norm. While social bias in language models has garnered a lot of attention in recent years, a majority of the existing work equates “bias” with “stereotypes”. We suggest reporting bias as an alternative lens through which to study how social attitudes manifest in language models. We present the MARB dataset, a diagnostic dataset for studying the interaction between social bias and reporting bias in language models. We use MARB to evaluate the off-the-shelf behavior of both masked and autoregressive language models and find signs of reporting bias with regards to marginalized identities, mirroring that which can be found in human text. This effect is particularly pronounced when taking gender into account, demonstrating the importance of considering intersectionality when studying social phenomena like biases.