Large language models (LLMs) are increasingly deployed over knowledge bases for efficient knowledge retrieval and question answering. However, LLMs can inadvertently answer beyond a user’s permission scope, leaking sensitive content and complicating deployment under fine-grained access-control requirements. We identify a geometric regularity in intermediate activations: representations associated with different permission scopes exhibit consistent and distinguishable structure in activation space. Building on this observation, we propose Activation-Space Anchored Access Control (AAAC), a training-free framework for multi-class permission control. AAAC constructs a compact bank of permission-specific activation anchors from a small offline calibration set and requires no model fine-tuning. At inference time, the trusted permission metadata is used by the controller to steer query activations toward authorized anchors and away from unauthorized ones, thereby mitigating the risk of over-privileged generation. Across three LLM families, AAAC achieves permission violation rates of 0.005–0.009 and prompt-based attack success rates of 0.084–0.101, while maintaining competitive utility with 10.3%–25.2% inference overhead.

Large language models have consistently struggled with complex reasoning tasks, such as mathematical problem-solving. Investigating the internal reasoning mechanisms of these models can help us design better model architectures and training strategies, ultimately enhancing their reasoning capability. In this study, we constructed a symbolic multi-step reasoning task to investigate the information propagation mechanisms in Transformer models when solving the task through direct answering and Chain-of-Thought (CoT) reasoning. We introduced the concept of buffer mechanism: the model stores various information in distinct buffers and selectively extracts it through the query-key matrix. We proposed a random matrix-based algorithm to enhance the model’s reasoning ability. This algorithm introduces only 132 trainable parameters, yet leads to significant performance improvements on 7 multi-step reasoning datasets, including PrOntoQA, LogicAsker, and LogicInference. These findings provide new insights into understanding the large language models.

Despite exceptional capabilities in knowledge-intensive tasks, Large Language Models (LLMs) face a critical gap in understanding how they internalize new knowledge, particularly how acquired knowledge becomes structurally embedded in their neural computations. We address this issue through the lens of knowledge circuit evolution, identifying computational subgraphs that facilitate knowledge storage and processing. Our systematic analysis of circuit evolution throughout continual pre-training reveals several key findings: (1) the acquisition of new knowledge is influenced by its relevance to pre-existing knowledge; (2) the evolution of knowledge circuits exhibits a distinct phase shift from formation to optimization; (3) the evolution of knowledge circuits follows a deep-to-shallow pattern. These insights not only advance our theoretical understanding of the mechanisms of new knowledge acquisition in LLMs, but also provide potential implications for improving continual pre-training strategies to enhance model performance.