HAL: Inducing Human-likeness in LLMs with Alignment

Masum Hasan, Junjie Zhao, Ehsan Hoque


Abstract
Aligning language models to qualitative behavioral traits, such as human-likeness, remains difficult because they are hard to define, measure, and optimize. As a result, improvements in human-like behavior are largely driven by scale or broad supervised training, rather than targeted alignment. We introduce Human Aligning LLMs (HAL), a framework for aligning language models to conversational human-likeness using an interpretable, data-driven reward. HAL derives explicit conversational traits from contrastive dialogue data, combines them into a compact scalar score, and uses this score as a transparent reward signal for alignment with standard preference optimization methods. Using this approach, we align models of varying sizes without affecting their overall performance. In large-scale Chatbot Arena-style human evaluations, a model aligned with HAL is more frequently perceived as human-like in conversation. Because HAL operates over explicit, interpretable traits, it enables inspection of alignment behavior and diagnosis of unintended effects. More broadly, HAL demonstrates how soft, qualitative properties of language–previously outside the scope for alignment–can be made measurable and aligned in an interpretable and explainable way.
Anthology ID:
2026.emnlp-main.1970
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
44834–44854
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1970/
DOI:
Bibkey:
Cite (ACL):
Masum Hasan, Junjie Zhao, and Ehsan Hoque. 2026. HAL: Inducing Human-likeness in LLMs with Alignment. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 44834–44854, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
HAL: Inducing Human-likeness in LLMs with Alignment (Hasan et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1970.pdf