Shaun Qien Yeau Tan
Author directory2026
The bias is in the generator: locating medical chain-of-thought subgroup gaps upstream of the verifier
Micky C. Nnamdi | Benoit Louis Marteau | Shaun Qien Yeau Tan | J. Ben Tamo | Yining Yuan | May Dongmei Wang
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Micky C. Nnamdi | Benoit Louis Marteau | Shaun Qien Yeau Tan | J. Ben Tamo | Yining Yuan | May Dongmei Wang
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Verifier-guided best-of-N decoding has two stages: a generator samples candidate chains of thought, and a verifier selects among them. A subgroup disparity may already be present in the generator’s candidates or be amplified during selection. We separate the two stages using adversarial demographic counterfactuals of medical exam questions. On HuatuoGPT-o1-8B, a substantial race accuracy gap is already present in a single sampled response, before verifier selection. Across seven generators, only Gemma-3-27B-it under Skywork-RM shows significant verifier-induced amplification after multiple-testing correction. Even an oracle selector leaves a residual gap on every medical generator analyzed, showing that correct answers are less often available for one subgroup. Counterfactually paired low-rank adaptation (LoRA) of the generator reduces the single-sample gap on two medical models, although on HuatuoGPT-o1-8B the reduction does not persist after self-judge selection. Single-variant controls indicate that the single-sample reduction is not explained by additional training steps alone. A short fairness instruction modestly reduces the single-sample gap, and a direct preference optimization (DPO) variant of the paired training also reduces it and, on HuatuoGPT-o1-8B, keeps race mention near its baseline rate. For the medical generators studied, candidate generation rather than inference-time selection is the primary locus for mitigating this gap.