Audio language models (LMs) are now integrated into high-stakes, decision-oriented settings: hiring pipelines, oral assessments, and language proficiency tests. Prior work in natural language processing has primarily focused on how LMs respond to socially-driven textual variation. With the growing use of LMs that natively process audio, similar robustness and fairness issues arise for speech variation. In our work, we examine how accent variation in English inputs affects audio LMs’ assessments of speakers. We evaluate six audio LMs on speech across five accent groups: American, British, Chinese, Indian, and Nigerian English. We find that several LMs assign lower speech delivery scores to Chinese- and Nigerian-accented speakers, regardless of content quality. At the individual level, delivery scores correlate negatively with acoustic distance from American English across three widely-used models (Spearman ρ = −0.35 to −0.66, all p < .05), suggesting that English speech is scored against an implicit American acoustic norm. Even though models are able to transcribe the speech in our studies, their assessment and perceptions of speakers may result in allocational harm. We release our framework and corpus to support future analyses of bias in audio LMs.

Large Language Models (LLMs) are increasingly employed in high-stakes decision-making tasks, such as loan approvals. While their applications expand across domains, LLMs struggle to process tabular data, ensuring fairness and delivering reliable predictions. In this work, we assess the performance and fairness of LLMs on serialized loan approval datasets from three geographically distinct regions: Ghana, Germany, and the United States. Our evaluation focuses on the model’s zero-shot and in-context learning (ICL) capabilities. Our results reveal that the choice of serialization format significantly affects both performance and fairness in LLMs, with certain formats such as GReaT and LIFT yielding higher F1 scores but exacerbating fairness disparities. Notably, while ICL improved model performance by 4.9-59.6% relative to zero-shot baselines, its effect on fairness varied considerably across datasets. Our work underscores the importance of effective tabular data representation methods and fairness-aware models to improve the reliability of LLMs in financial decision-making.

Africa has a very poor doctor-to-patient ratio. At very busy clinics, doctors could see 30+ patients per day—a heavy patient burden compared with developed countries—but productivity tools such as clinical automatic speech recognition (ASR) are lacking for these overworked clinicians. However, clinical ASR is mature, even ubiquitous, in developed nations, and clinician-reported performance of commercial clinical ASR systems is generally satisfactory. Furthermore, the recent performance of general domain ASR is approaching human accuracy. However, several gaps exist. Several publications have highlighted racial bias with speech-to-text algorithms and performance on minority accents lags significantly. To our knowledge, there is no publicly available research or benchmark on accented African clinical ASR, and speech data is non-existent for the majority of African accents. We release AfriSpeech, 200hrs of Pan-African English speech, 67,577 clips from 2,463 unique speakers across 120 indigenous accents from 13 countries for clinical and general domain ASR, a benchmark test set, with publicly available pre-trained models with SOTA performance on the AfriSpeech benchmark.