Knowledge editing aims to update specific facts in language models to ensure reliable and up-to-date behavior. A key challenge is whether edited knowledge generalizes beyond the format in which the edit is applied, yet existing evaluations primarily assess generalization using paraphrases. In this work, we identify a critical blind spot: generalization across heterogeneous task formats, including multiple-choice and verification-based question answering. We show that paraphrase-based evaluations overestimate robustness, as models that perform well on paraphrases exhibit substantial performance drops across task formats. While model predictions are consistent across formats prior to editing, editing introduces format-dependent inconsistencies. We further observe that heterogeneous task formats exhibit distinct decision landscapes, where single-format objectives lead to format-specific local optima with divergent edit directions, resulting in misaligned outputs across formats. To address this issue, we demonstrate that aligning edit directions across task formats improves cross-format generalization and consistency. These findings highlight the need to move beyond paraphrase-based evaluation and to develop editing methods that maintain consistent factual behavior across diverse task formulations.

Large language models (LLMs) have demonstrated remarkable capabilities in natural language processing tasks. However, they remain vulnerable to semantic inconsistency, where minor formatting variations result in divergent predictions for semantically equivalent inputs. Our comprehensive evaluation reveals that this brittleness persists even in state-of-the-art models such as GPT-4o, posing a serious challenge to their reliability. Through a mechanistic analysis, we find that semantic-equivalent input changes induce instability in internal representations, ultimately leading to divergent predictions. This reflects a deeper structural issue, where form and meaning are intertwined in the embedding space. We further demonstrate that existing mitigation strategies, including direct fine-tuning on format variations, do not fully address semantic inconsistency, underscoring the difficulty of the problem. Our findings highlight the need for deeper mechanistic understanding to develop targeted methods that improve robustness.