DanDan Liu

Author directory

2026

Large language models (LLMs) are increasingly used as virtual survey respondents, yet most evaluations compare only item-level marginal response distributions. Such evaluations can miss two failures central to computational social science: synthetic respondents may distort correlations among attitudes, and errors may be distributed unevenly across countries or demographic groups. We introduce FairSynth, an instrument-relative framework for auditing and calibrating closed-ended, answer-only LLM survey simulations against existing anchor surveys. FairSynth evaluates marginal fidelity, Structural Attitude Fidelity (item–item correlation similarity), group representation error, distributional flattening, and between-group sign reversals. It then applies a model-agnostic post-hoc logit-shift layer with shared group parameters to reduce disparities in representation error while preserving attitude structure. Experiments on World Values Survey Wave 7 and General Social Survey 2022 show that FairSynth reduces worst-group representation error and sign reversals while improving structural fidelity on the evaluated questionnaires. FairSynth is not a substitute for human surveys; it is a measurement-fairness toolkit for auditing, stress-testing, and calibrating synthetic survey simulations when anchor data are available.
When asked explicitly, a Large language model (LLM) may validate your anger—but implicitly, it may still judge that anger as inappropriate. We call this divergence the endorsement–exposure gap, and it reveals that LLMs encode hidden norms about which emotions are acceptable in which contexts. To measure these norms systematically, we introduce Feeling Rules Atlas, a benchmark of 1,320 vignettes spanning 6 institutional settings, 12 roles, 7 emotions, and 5 intensity levels. We pair the benchmark with two probes: explicit norm judgments (APPROPRIATE/INAPPROPRIATE/DEPENDS) and implicit acceptability scored by log-likelihood contrast. Across six model families, we find large cross-model variation in sanctioning thresholds and institutional “norm signatures” not reducible to overall strictness; models that appear similarly lenient explicitly can diverge sharply in implicit judgments. These results establish normative affect—context-conditioned judgments of emotional appropriateness—as a distinct alignment axis, and motivate transparent profiling of feeling rules for emotionally sensitive deployments.