Large language models persistently produce literal translations of idiomatic expressions, but the source of this failure remains unclear. We introduce **IdiomDIT**, a diagnostic framework that jointly evaluates idiom **D**etection, idiom **I**nterpretation, and idiomatic **T**ranslation, attributing each error to a specific source. Across six language pairs and five models spanning 4B to 70B, IdiomDIT shows that literal translation error (LTE) can persist even after successful detection and interpretation, a know-but-error pattern we term Literal Translation Bias (LTB). This pattern is direction-specific: LTB dominates some directions, while others show greater knowledge deficits, likely reflecting lower idiom exposure in pretraining. We then use linear probing to test whether LTB is already reflected in the model’s pre-generation hidden state, recovering per-language literal translation directions with high accuracy, and ablating this direction reduces LTE. However, once we ensure no idiom’s data leaks across training and test, and subtract out each idiom’s mean hidden state before probing, accuracy collapses, confirming the probe was largely detecting which idiom it is looking at rather than a genuine bias signal. A matched random direction also reduces LTE just as much as the identified one. We release these controls as a reusable protocol for testing whether such hidden-state attribution claims in machine translation are genuine or confounded. The code is publicly available at https://github.com/NLP2CT/IdiomDIT.

Idioms are difficult to transfer across languages due to their non-compositionality and weak surface-form grounding, making literal mappings unreliable. We present G-IdiomAlign, a gloss-pivoted benchmark where each idiom is anchored by an English gloss from Wiktionary. We further construct a high-confidence reference alignment set for reproducible evaluation. G-IdiomAlign supports two protocols: (1) a controlled Multiple-Choice Idiom Equivalence with typed distractors for error attribution; and (2) a Gloss-Contrastive Generation contrasting No-gloss and With-gloss inputs to isolate the effect of an explicit semantic pivot. Across diverse LLMs, a bias to literal translation is a dominant failure mode, especially when the target is a low-resource language. Glosses consistently improve Gloss-Contrastive Generation under an embedding-based semantic proxy, but performance remains modest, indicating substantial headroom in the open output space. Subsequent analysis on Qwen3-8B further suggests that cross-condition differences are concentrated more in attention heads than in layers, while better With-gloss generations coincide with stronger gloss anchoring.

Recent pre-trained language models (PLMs) achieve promising results in existing abstractive summarization datasets. However, existing summarization benchmarks overlap in time with the standard pre-training corpora and finetuning datasets. Hence, the strong performance of PLMs may rely on the parametric knowledge that is memorized during pre-training and fine-tuning. Moreover, the knowledge memorized by PLMs may quickly become outdated, which affects the generalization performance of PLMs on future data. In this work, we propose TempoSum, a novel benchmark that contains data samples from 2010 to 2022, to understand the temporal generalization ability of abstractive summarization models. Through extensive human evaluation, we show that parametric knowledge stored in summarization models significantly affects the faithfulness of the generated summaries on future data. Moreover, existing faithfulness enhancement methods cannot reliably improve the faithfulness of summarization models on future data. Finally, we discuss several recommendations to the research community on how to evaluate and improve the temporal generalization capability of text summarization models.