Jooyoung Kim

Author directory

2026

Social-bias benchmarks such as BBQ (Parrish et al., 2022) evaluate whether language models rely on stereotypes in ambiguous contexts where the correct answer indicates insufficient information. We observe that BBQ’s ambiguous-context bias scores aggregate over two distinct phenomena: abstention failure (selecting any specific person despite insufficient evidence) and stereotype direction (favoring the stereotype-labeled option among those unsupported selections). To formalize this distinction, we introduce a decomposition that factors the raw stereotype-consistent choice rate into a non-abstention rate and a conditional stereotype rate—the two quantities that underlie BBQ’s standard score. Across 11 BBQ categories, answerability prompting substantially reduces raw stereotype-consistent choices (0.068 to 0.023), but our decomposition shows that this is driven almost entirely by improved abstention (non-abstention 0.166 to 0.053), while the conditional stereotype rate does not fall consistently and rises in Disability Status and Physical Appearance. Cross-model evaluation on four proprietary and open-weight LLMs confirms the same pattern, and counterfactual surface-cue controls on Age, Disability Status, and a Physical Appearance subset further show that concrete descriptors account for a substantial share of abstention failures while stereotype direction persists even at minimal group labels. These results demonstrate that disentangling abstention failure from stereotype direction yields a more informative, diagnostic view of model behavior under ambiguity.
Language model (LM)-powered agents have shown promise in addressing complex embodied tasks by leveraging the commonsense world knowledge of pre-trained models in interactive physical environments. Yet, real-world deployment requires agents to continually adapt to a stream of tasks, where neither large-scale expert supervision nor reliable external feedback is consistently available. In this paper, we present ASC, a framework for efficient continual embodied task adaptation through amortizing self-correction. Specifically, ASC trains the policy under two objectives: a standard CoT reasoning objective distilled from stronger experts, and a reflexive reasoning objective that internalizes self-correction by correcting initial rationales generated by a previously learned policy. This dual optimization allows the agent to adapt more efficiently through forward knowledge transfer over time, ultimately fostering robust CoT reasoning at inference without requiring explicit reflection. Furthermore, we introduce rationale-wise test-time scaling that dynamically adjusts the depth of CoT reasoning based on the model’s confidence in actions inferred from its own rationales. Through experiments on VirtualHome and ALFWorld, we demonstrate performance superiority over other LM-based planning and continual task adaptation approaches, while achieving strong efficiency in computation, data usage.