The remarkable performance of Mixture-of-Experts (MoE) models often comes with substantial architectural scale and resource overhead. Prior structural compression mainly focuses on expert-level or FFN intermediate pruning. However, we identify an emerging architectural trend that the FFN expansion ratio in state-of-the-art MoEs has steadily decreased, making intermediate-only pruning less effective and motivating us to explore other redundancies. Motivated by this, we propose an orchestrated hidden-intermediate prune-and-distill framework that, to the best of our knowledge, is the first to jointly prune expert FFNs along both intermediate and hidden dimensions. It includes Budget-Constrained Coordinate Descent to efficiently search expert-specific pruning ratios across the two dimensions, and attention relation transfer distillation to provide stable supervision under hidden misalignment. Extensive experiments across diverse benchmarks validates that hidden-dimension pruning is a promising direction for compressing modern MoEs.

Dynamic perioperative risk prediction requires models to anticipate evolving physiological deterioration under strict prediction-time information boundaries. We introduce PERIBENCH, a protocol-aware benchmark built from high-frequency intraoperative signals. PERIBENCH defines 18 clinically motivated tasks spanning future risk classification, persistent deterioration prediction, continuous vital-sign regression, and case-level prognosis, and evaluates 37 systems across numerical predictors, deep sequential models, time-series foundation models, and LLM-oriented language interfaces. Rather than using a single undifferentiated leaderboard, PERIBENCH separates supervised or adapted prediction, direct pretrained transfer, and prompt-based language-interface inference into protocol-compatible pools. Experiments show that model sophistication does not translate monotonically into better perioperative prediction: tree-based references remain strong on many classification endpoints, deep sequential models are competitive on selected physiological tasks, direct foundation-model transfer is heterogeneous, and language interfaces capture useful classification signals but remain less reliable for regression, calibration, alert burden, and abnormal-region prediction. PERIBENCH provides a structured benchmark for studying dynamic perioperative prediction from numerical models to language interfaces under shared task semantics and explicit protocol boundaries. Code is available at https://github.com/Kdyyh/PERIBENCH.

Long-context reasoning has significantly empowered large language models to tackle complex tasks, yet it introduces severe efficiency bottlenecks due to the computational complexity. Existing efficient approaches often rely on complex additional training or external models for compression, which limits scalability and discards critical fine-grained information. In this paper, we first propose VTC-R1, a new efficient reasoning paradigm that integrates vision-text compression into iterative reasoning. Instead of processing lengthy textual traces, VTC-R1 renders intermediate reasoning segments into compact images, which are iteratively fed back into vision-language models as "optical memory". We construct a training dataset based on OpenR1-Math-220K achieving 3.4× token compression and fine-tune representative VLMs–Glyph and Qwen3-VL. Extensive experiments on benchmarks such as MATH500, AIME25, AMC23 and GPQA-D demonstrate that VTC-R1 consistently outperforms standard long-context reasoning. And VTC-R1 improves inference efficiency, achieving 2.7× speedup in end-to-end latency, highlighting its potential as a scalable solution for reasoning-intensive applications.

Knowledge distillation (KD) transfers capabilities from large language models (LLMs) to smaller students, yet it can fail unpredictably and also underpins model leakage risks. Our analysis revealed several distillation traps: tail noise, off-policy instability, and, most fundamentally, the teacher–student gap, that distort training signals. These traps manifest as overconfident hallucinations, self-correction collapse, and local decoding degradation, causing distillation to fail. Motivated by these findings, we propose a post-hoc calibration method that, to the best of our knowledge, for the first time enables control over a teacher’s distillability via reinforcement fine-tuning (RFT). Our objective combines task utility, KL anchor, and across-tokenizer calibration reward. This makes distillability a practical safety lever for foundation models, connecting robust teacher–student transfer with deployment-aware model protection. Experiments across math, knowledge QA, and instruction-following tasks show that students distilled from distillable calibrated teachers outperform SFT and KD baselines, while undistillable calibrated teachers retain their task performance but cause distilled students to collapse, offering a practical knob for both better KD and model IP protection.

In this paper, we investigate knowledge forgetting in large language models with a focus on its generalisation—ensuring that models forget not only specific training samples but also related implicit knowledge. To this end, we begin by identifying a broader unlearning scope that includes both target data and logically associated samples, including rephrased, subject-replaced, relation-reversed, and one-hop reasoned data. We then conduct a rigorous evaluation of 15 state-of-the-art methods across three datasets, revealing that unlearned models still recall paraphrased answers and retain target facts in their intermediate layers. This motivates us to take a preliminary step toward more generalised implicit knowledge forgetting by proposing PERMU—a novel probability perturbation-based unlearning paradigm. PERMU simulates adversarial unlearning samples to eliminate fact-related tokens from the logit distribution, collectively reducing the probabilities of all answer-associated tokens. Experiments are conducted on a diverse range of datasets, including TOFU, Harry Potter, ZsRE, WMDP, and MUSE, using models ranging from 1.3B to 13B in scale. The results demonstrate that PERMU delivers up to a 50.40% improvement in unlearning vanilla target data while maintaining a 40.73% boost in forgetting implicit knowledge. Our code can be found in the supplementary material.

Tree of Thoughts (ToT) enhances Large Language Model (LLM) reasoning by structuring problem-solving as a spanning tree. However, recent methods focus on search accuracy while overlooking computational efficiency. The challenges of accelerating the ToT lie in the frequent switching of reasoning focus, and the redundant exploration of suboptimal solutions. To alleviate this dilemma, we propose Dynamic Parallel Tree Search (DPTS), a novel parallelism framework that aims to dynamically optimize the reasoning path in inference. It includes the Parallelism Streamline in the generation phase to build up a flexible and adaptive parallelism with arbitrary paths by cache management and alignment. Meanwhile, the Search and Transition Mechanism filters potential candidates to dynamically maintain the reasoning focus on more possible solutions with less redundancy. Experiments on Qwen-2.5 and Llama-3 on math and code datasets show that DPTS significantly improves efficiency by 2-4× on average while maintaining or even surpassing existing reasoning algorithms in accuracy, making ToT-based reasoning more scalable and computationally efficient. Codes are released at: https://github.com/yifu-ding/DPTS.