SHARD: Safe and Helpful Alignment via Self-Reframing Distillation

Viswonathan Manoranjan, Amogh Gupta, Anvesh Rao Vijjini, Thomas Hofweber, Snigdha Chaturvedi


Abstract
Large language models often struggle with sensitive prompts. They may refuse outright, provide generic safety boilerplate, or fail to address the user’s legitimate informational needs that can be answered safely. We introduce SHARD, a self-reframing distillation method to improve safety-helpfulness. It first rewrites sensitive prompts to surface benign intent using philosophical guidelines, then reframes its original responses into safe, more helpful ones, and finally fine-tunes the model on its self-reframed responses. Across DNA and the English subset of LINGUASAFE, SHARD improves helpfulness for most model families while preserving safety. It also remains competitive with distillation from a larger teacher model, suggesting that models can internalize safe and helpful behavior elicited from the models themselves.
Anthology ID:
2026.findings-emnlp.1460
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
31076–31103
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1460/
DOI:
Bibkey:
Cite (ACL):
Viswonathan Manoranjan, Amogh Gupta, Anvesh Rao Vijjini, Thomas Hofweber, and Snigdha Chaturvedi. 2026. SHARD: Safe and Helpful Alignment via Self-Reframing Distillation. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 31076–31103, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
SHARD: Safe and Helpful Alignment via Self-Reframing Distillation (Manoranjan et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1460.pdf