Low-Rank Adaptation (LoRA) is widely used for parameter-efficient fine-tuning, yet merging multiple task-specific LoRA updates without additional training remains challenging. Most existing LoRA merging methods rely on SVD-based alignment, which emphasizes globally shared structure across tasks. In this work, we show that LoRA merging performance can be further improved by combining SVD with CUR decomposition. Through a representation-level analysis, we find that SVD-based decompositions primarily model shared components across tasks, while CUR-based decompositions better preserve task-specific and localized updates. These two perspectives are geometrically misaligned and exhibit complementary advantages, revealing an inherent trade-off between capturing shared structure and preserving task-specific information in LoRA model merging. Guided by this analysis, we propose a training-free merging procedure that explicitly combines the shared structure captured by SVD with the task-specific components preserved by CUR. Experiments on both vision and language benchmarks demonstrate consistent improvements over existing gradient-free LoRA merging methods.

Syntactic prediction requires models to generate outputs that are globally consistent and satisfy hierarchical structural constraints. Although large language models have been applied to such tasks, existing approaches still rely mainly on prompting or supervised fine-tuning, and their performance generally remains below that of specialized neural syntactic models. Our experiments show that supervised fine-tuning reaches a plateau, and additional supervised training does not further improve prediction quality. We therefore propose a two-stage post-training framework for syntactic prediction. The first cold-start stage adapt the model to a unified syntax generation format, and then applies reinforcement learning on a disjoint data subset to directly optimize structure-level objectives. Experiments on constituency parsing and CCG supertagging show that Syntax-RL consistently outperforms supervised LLM baselines and achieves state-of-the-art results. Further analyses suggest that its gains come primarily from the reinforcement learning stage. These findings identify reinforcement learning as an effective post-training mechanism for improving hierarchical syntactic prediction in large language models.