DanDan Liu
Author directory2026
FairSynth: Instrument-Relative Evaluation and Calibration of LLM Survey Simulations for Measurement Fairness
Guangrui Fan | DanDan Liu | Qian Guo | Xueying Niu | Lihu Pan
Findings of the Association for Computational Linguistics: EMNLP 2026
Guangrui Fan | DanDan Liu | Qian Guo | Xueying Niu | Lihu Pan
Findings of the Association for Computational Linguistics: EMNLP 2026
Large language models (LLMs) are increasingly used as virtual survey respondents, yet most evaluations compare only item-level marginal response distributions. Such evaluations can miss two failures central to computational social science: synthetic respondents may distort correlations among attitudes, and errors may be distributed unevenly across countries or demographic groups. We introduce FairSynth, an instrument-relative framework for auditing and calibrating closed-ended, answer-only LLM survey simulations against existing anchor surveys. FairSynth evaluates marginal fidelity, Structural Attitude Fidelity (item–item correlation similarity), group representation error, distributional flattening, and between-group sign reversals. It then applies a model-agnostic post-hoc logit-shift layer with shared group parameters to reduce disparities in representation error while preserving attitude structure. Experiments on World Values Survey Wave 7 and General Social Survey 2022 show that FairSynth reduces worst-group representation error and sign reversals while improving structural fidelity on the evaluated questionnaires. FairSynth is not a substitute for human surveys; it is a measurement-fairness toolkit for auditing, stress-testing, and calibrating synthetic survey simulations when anchor data are available.
Feeling Rules in Language Models: Mapping Norms of Emotional Appropriateness Across Roles, Institutions, and Intensity
Guangrui Fan | DanDan Liu | Aznul Qalid MD Sabri | Rui Zhang | Pan Lihu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Guangrui Fan | DanDan Liu | Aznul Qalid MD Sabri | Rui Zhang | Pan Lihu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
When asked explicitly, a Large language model (LLM) may validate your anger—but implicitly, it may still judge that anger as inappropriate. We call this divergence the endorsement–exposure gap, and it reveals that LLMs encode hidden norms about which emotions are acceptable in which contexts. To measure these norms systematically, we introduce Feeling Rules Atlas, a benchmark of 1,320 vignettes spanning 6 institutional settings, 12 roles, 7 emotions, and 5 intensity levels. We pair the benchmark with two probes: explicit norm judgments (APPROPRIATE/INAPPROPRIATE/DEPENDS) and implicit acceptability scored by log-likelihood contrast. Across six model families, we find large cross-model variation in sanctioning thresholds and institutional “norm signatures” not reducible to overall strictness; models that appear similarly lenient explicitly can diverge sharply in implicit judgments. These results establish normative affect—context-conditioned judgments of emotional appropriateness—as a distinct alignment axis, and motivate transparent profiling of feeling rules for emotionally sensitive deployments.