Parameter-efficient fine-tuning (PEFT) methods like LoRA have become crucial for adapting large language models under computational constraints. While LoRA focuses on dense weight updates, recent sparse methods propose updating only selected portions of pretrained weights. However, these sparse approaches predominantly use element-wise parameter selection, overlooking inherent structured patterns in LLM weight distributions. We introduce StructTune1, a PEFT method that uses entire rows or columns as basic units for identifying task-relevant sparse components. Our analysis reveals LLaMA models exhibit distinct structured patterns in pretrained weights, with fine-tuning updates showing structured amplification rather than uniform changes. We develop a sensitivity-based framework to efficiently identify task-relevant weight structures without full fine-tuning. Experiments on mathematical reasoning and code generation demonstrate StructTune’s structured sparse updates outperform both dense LoRA and element-wise sparse methods while maintaining computational efficiency. Ablation studies validate our sensitivity-based selection strategy, confirming the efficacy of exploiting structured weight patterns in PEFT.

Monte Carlo Tree Search (MCTS) is a promising test-time method for improving LLMs on hard reasoning problems, where correct trajectories are often sparse. Self-evaluated MCTS is attractive because it avoids external reward models, verifiers, and stronger judges, but it faces a central paradox: MCTS depends on reliable intermediate rewards, while LLM self-evaluation is noisy and unstable. We propose ASE-MCTS, an Asynchronous Self-Evaluated MCTS framework. ASE-MCTS uses search-oriented reward analysis to derive an effective composition of rollout, process, and pairwise rewards from their global and local search roles. It further combines flexible search-space control with dependency-aware asynchronous execution, reducing redundant exploration and latency. Experiments on mathematical and scientific reasoning benchmarks show that ASE-MCTS improves reasoning without external supervision, especially on hard sparse-trajectory problems, and achieves up to a 6× wall-clock speedup over Sync-MCTS in the branch-factor scalability experiment.

Large language models (LLMs) have demonstrated significant reasoning capabilities in scientific discovery but struggle to bridge the gap to physical execution in wet-labs. In these irreversible environments, probabilistic hallucinations are not merely incorrect; they can cause equipment damage or experimental failure. To address this, we propose BioProAgent, a neuro-symbolic framework that anchors probabilistic planning in a deterministic Finite State Machine (FSM). We introduce a State-Augmented Planning mechanism that enforces a rigorous Design-Verify-Rectify workflow, ensuring hardware compliance before execution. Furthermore, we address the context bottleneck inherent in complex device schemas by Semantic Symbol Grounding, reducing token consumption by ∼6× through symbolic abstraction. In the extended BioProBench benchmark, BioProAgent achieves 95.6% physical compliance (compared to 21.0% for ReAct), demonstrating that neuro-symbolic constraints are essential for reliable autonomy in irreversible physical environments. Code at https://github.com/YuyangSunshine/bioproagent and Website at https://yuyangsunshine.github.io/BioPro-Project/ .

The opaque nature of neural networks, often described as black boxes, poses significant challenges in understanding their learning mechanisms, which limit our ability to fully optimize and trust these models.Inspired by how humans learn, this paper proposes a novel neural network training strategy that employs multitask learning with progressive difficulty subtasks, which we believe can potentially shed light on the internal learning mechanisms of neural networks.We implemented this strategy across a range of NLP tasks, data sets, and neural network architectures and observed notable improvements in model performance.This suggests that neural networks may be able to extract common features and internalize shared representations across similar subtasks that differ in their difficulty.Analyzing this strategy could lead us to more interpretable and robust neural networks, enhancing both their performance and our understanding of their nature.

In this study, we explore the performance oflarge language models (LLMs) using differ-ent prompt engineering approaches in the con-text of legal text classification. Prior researchhas demonstrated that various prompting tech-niques can improve the performance of a di-verse array of tasks done by LLMs. However,in this research, we observe that professionaldocuments, and in particular legal documents,pose unique challenges for LLMs. We experi-ment with several LLMs and various promptingtechniques, including zero/few-shot prompting,prompt ensembling, chain-of-thought, and ac-tivation fine-tuning and compare the perfor-mance on legal datasets. Although the newgeneration of LLMs and prompt optimizationtechniques have been shown to improve gener-ation and understanding of generic tasks, ourfindings suggest that such improvements maynot readily transfer to other domains. Specifi-cally, experiments indicate that not all prompt-ing approaches and models are well-suited forthe legal domain which involves complexitiessuch as long documents and domain-specificlanguage.