Text embeddings are central to modern natural language processing, enabling several downstream tasks. Despite their significance, existing evaluation frameworks primarily target English and other high-resource languages, leaving critical gaps for languages such as Turkish. To address this, we present TR-TEB (Turkish Text Embedding Benchmark), the first comprehensive, standardized, and reproducible benchmark for Turkish text embeddings. TR-TEB spans five core task categories: classification, pair classification, clustering, retrieval, and semantic textual similarity. It is supported by a diverse dataset portfolio that integrates 14 curated open-source resources, 26 high-quality translated datasets, and 7 newly constructed Turkish-specific datasets designed to capture the language’s unique characteristics. We test our framework by comparing 45 well-known open-source embedding models. As the first unified evaluation suite, TR-TEB serves as a core tool for the Turkish embedding research community, establishing a systematic basis for model comparison and improvement. Furthermore, its benchmarking methodology and dataset creation process provide a blueprint for extending robust embedding evaluation to other low-resource languages.

Large language models (LLMs) are being explored as decision-support tools for financial risk assessment. Their outputs may, however, respond to demographic descriptions that are unrelated to the underlying financial record. We study this behavior through a paired counterfactual audit. The same 100 commercial credit and cheque-payment profiles are evaluated repeatedly, with only one demographic descriptor changed at a time. The experiments cover 7 model variants, three prompting strategies, and descriptors related to gender, education, and citizenship. While paired gender score differences remain within a ±1.5 point equivalence margin in 12 of 21 model–strategy cells, education shows a strong, unambiguous effect. Risk decreases monotonically with education level (103 of 126 pairwise contrasts remain significant after Holm correction, score differences reaching 26.0 points between primary-school and doctorate descriptors), it flips up to 42% of approve/reject decisions, and elicited rationales name education as a risk factor even though education is not a financially legitimate predictor. Citizenship produces larger shifts still, flipping up to 51% of decisions while leaving AUC almost unchanged in most cells, and in one model weakening ranking (0.794 to 0.438). Similar to education, rationales name the descriptor as the reason. Unlike citizenship, whose jurisdictional cues can legitimately enter sanctions or AML/CFT screening, the education effect has no such justification, making it the clearest evidence of demographic bias. A follow-up experiment in English, Chinese, and Arabic shows that citizenship sensitivity depends on both prompt language and prompting strategy, affecting the magnitude and direction of score shifts. The study provides a standardized counterfactual framework for auditing LLM-generated scores before they are considered for high-stakes financial workflows.