Maurice Pagnucco
Author directory2026
Identifying Temporal Features within Transcoders for Time Sensitive Factual Recall
Sanjay Govindan | Maurice Pagnucco | Yang Song
Findings of the Association for Computational Linguistics: EMNLP 2026
Sanjay Govindan | Maurice Pagnucco | Yang Song
Findings of the Association for Computational Linguistics: EMNLP 2026
Large Language Models (LLMs) suffer from temporal misalignment, often due to the contradictory nature of their training corpora. While current mitigation strategies rely on computationally expensive fine-tuning or context-heavy retrieval augmented generation (RAG), the internal mechanisms governing time sensitive recall remain under-explored. Unlike prior studies that identify temporal components such as attention heads and MLP layers, we provide the first feature level map of temporal recall by isolating individual MLP features via transcoder circuit tracing. We identify three node categories (common temporal, common to the year and chrono-semantic) which interact to generate a temporal filter during factual recall. By analysing Gemma 2 2B, LLaMA 3.2 1B, and Qwen3-4b, we show that these features do not follow a simple linear pipeline but represent time through a parallel and mixed syntactic-semantic interplay across layers. We additionally discover a class of higher-layer temporal components invisible to existing EAP-IG methods, establishing transcoders as a more complete lens for temporal interpretability for time sensitive factual recall. These findings present MLP components for potential targeted interventions in time-sensitive factual recall. Our code can be found at https://github.com/sg-sy/timewarp-temporaltranscoder.
2025
Temporal Alignment of Time Sensitive Facts with Activation Engineering
Sanjay Govindan | Maurice Pagnucco | Yang Song
Findings of the Association for Computational Linguistics: EMNLP 2025
Sanjay Govindan | Maurice Pagnucco | Yang Song
Findings of the Association for Computational Linguistics: EMNLP 2025
Large Language Models (LLMs) are trained on diverse and often conflicting knowledge spanning multiple domains and time periods. Some of this knowledge is only valid within specific temporal contexts, such as answering the question, “Who is the President of the United States in 2022?” Ensuring LLMs generate time-appropriate responses is crucial for maintaining relevance and accuracy. In this work we explore activation engineering as a method for temporally aligning LLMs to improve factual recall without any training. Activation engineering has predominantly been used to steer subjective and qualitative outcomes such as toxicity or behavior. Our research is one of few that uncovers the bounds of activation engineering on objective outcomes. We explore an activation engineering technique to anchor LLaMA 2, LLaMA 3.1, Qwen 2 and Gemma 2 to specific points in time and examine the effects of varying injection layers and prompting strategies. Our experiments demonstrate up to a 44% and 16% improvement in relative and explicit prompting respectively, achieving comparable performance to the fine-tuning method proposed by Zhao et al. (2024). Notably, for LLaMA 2 and LLaMA 3.1 our approach achieves similar results to the fine-tuning baseline while being significantly more computationally efficient and requiring no pre-aligned datasets.