As large language models (LLMs) are deployed globally, creating pluralistic systems that can accommodate the diverse preferences and values of users becomes essential. We introduce EVALUESTEER, a human-validated benchmark to measure the steerability of LLM and reward models (RMs) to users’ value and stylistic preference profiles grounded in psychology and literature on human-LLM interaction. To address the gap in existing datasets that do not support controlled evaluations of RM steering, we synthetically generated 165,888 preference pairs – systematically varying pairs along 4 value dimensions (traditional, secular-rational, survival, and self-expression) and 4 style dimensions (verbosity, readability, confidence, warmth). We use EVALUESTEER to evaluate whether, given a user profile and a pair of candidate value and style laden responses, LLMs and RMs are able to select the output that aligns with the user’s preferences. Our results show that, when given the user’s full profile of values and stylistic preferences, the best models achieve <75% accuracy in choosing the correct response, in contrast to > 99% accuracy when only relevant style and value preferences are provided. RMs also exhibit systematic biases, favoring secular values and verbose, confident language, and prioritizing style alignment over value alignment when the two conflict. EVALUESTEER thus highlights the limitations of current RMs at identifying and adapting to relevant user profile information, and provides a challenging testbed for developing RMs that can be steered towards diverse human values and preferences.

Culture moderates the way individuals perceive and express mental distress. Current understandings of mental health expressions on social media, however, are predominantly derived from WEIRD (Western, Educated, Industrialized, Rich, and Democratic) contexts. To address this gap, we examine mental health posts on Reddit made by individuals geolocated in India, to identify variations in social media language specific to the Indian context compared to users from Western nations. Our experiments reveal significant psychosocial variations in emotions and temporal orientation. This study demonstrates the potential of social media platforms for identifying cross-cultural differences in mental health expressions (e.g. seeking advice in India vs seeking support by Western users). Significant linguistic variations in online mental health-related language emphasize the importance of developing precision-targeted interventions that are culturally appropriate.

In this paper, we analyze zero-shot taxonomy learning methods which are based on distilling knowledge from language models via prompting and sentence scoring. We show that, despite their simplicity, these methods outperform some supervised strategies and are competitive with the current state-of-the-art under adequate conditions. We also show that statistical and linguistic properties of prompts dictate downstream performance.