<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kashyap-etal-2026-alignment">
    <titleInfo>
        <title>Is Alignment Enough for Helpfulness, Harmlessness, and Honesty?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Gautam</namePart>
        <namePart type="given">Siddharth</namePart>
        <namePart type="family">Kashyap</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mark</namePart>
        <namePart type="family">Dras</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Usman</namePart>
        <namePart type="family">Naseem</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Aligning Large Language Models (LLMs) with the objectives of helpfulness, harmlessness, and honesty (HHH) remains important for reliable deployment. Existing approaches commonly rely on Supervised Fine-Tuning (SFT) and Mixture-of-Experts (MoE) for alignment. Prior work has identified Axis Collapse as a failure mode where different alignment objectives occupy disjoint feature spaces, with SFT showing interference between conflicting objectives, i.e. catastrophic forgetting, while MoE models show miscalibrated routing due to unreliable expert selection. Reinforcement Learning from Human Feedback (RLHF) has been applied to mitigate this, though it frequently shows reward misattribution, where alignment signals disproportionately favor certain objectives over others. To address these issues, we propose TriAlignX, a two-stage framework for HHH alignment. In Stage 1, low-rank task vectors are extracted for each alignment objective and routed through PrefSelect, a query-conditioned policy that selects relevant objectives for each query. In Stage 2, an encoding module maps the selected task vectors into objective-specific alignment scores, while a response module produces alignment-aware embeddings for decoding. Experimental results show that TriAlignX achieves up to 97.85% helpfulness (↑), 26.65% harmlessness (↓), and 98.90% honesty (↑), with an Avg score of 55.65%. On DeepSeek-7B, TriAlignX further achieves 54.96% Avg while reducing inference latency to 0.68s and peak memory usage to 13.2GB. Our code is available at: https://github.com/gskgautam/TriAlignX</abstract>
    <identifier type="citekey">kashyap-etal-2026-alignment</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.2294/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>52189</start>
            <end>52201</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Is Alignment Enough for Helpfulness, Harmlessness, and Honesty?
%A Kashyap, Gautam Siddharth
%A Dras, Mark
%A Naseem, Usman
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F kashyap-etal-2026-alignment
%X Aligning Large Language Models (LLMs) with the objectives of helpfulness, harmlessness, and honesty (HHH) remains important for reliable deployment. Existing approaches commonly rely on Supervised Fine-Tuning (SFT) and Mixture-of-Experts (MoE) for alignment. Prior work has identified Axis Collapse as a failure mode where different alignment objectives occupy disjoint feature spaces, with SFT showing interference between conflicting objectives, i.e. catastrophic forgetting, while MoE models show miscalibrated routing due to unreliable expert selection. Reinforcement Learning from Human Feedback (RLHF) has been applied to mitigate this, though it frequently shows reward misattribution, where alignment signals disproportionately favor certain objectives over others. To address these issues, we propose TriAlignX, a two-stage framework for HHH alignment. In Stage 1, low-rank task vectors are extracted for each alignment objective and routed through PrefSelect, a query-conditioned policy that selects relevant objectives for each query. In Stage 2, an encoding module maps the selected task vectors into objective-specific alignment scores, while a response module produces alignment-aware embeddings for decoding. Experimental results show that TriAlignX achieves up to 97.85% helpfulness (↑), 26.65% harmlessness (↓), and 98.90% honesty (↑), with an Avg score of 55.65%. On DeepSeek-7B, TriAlignX further achieves 54.96% Avg while reducing inference latency to 0.68s and peak memory usage to 13.2GB. Our code is available at: https://github.com/gskgautam/TriAlignX
%U https://aclanthology.org/2026.emnlp-main.2294/
%P 52189-52201

Markdown (Informal)

[Is Alignment Enough for Helpfulness, Harmlessness, and Honesty?](https://aclanthology.org/2026.emnlp-main.2294/) (Kashyap et al., EMNLP 2026)

• Is Alignment Enough for Helpfulness, Harmlessness, and Honesty? (Kashyap et al., EMNLP 2026)

ACL

• Gautam Siddharth Kashyap, Mark Dras, and Usman Naseem. 2026. Is Alignment Enough for Helpfulness, Harmlessness, and Honesty?. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 52189–52201, Budapest, Hungary. Association for Computational Linguistics.