Singular Value Decomposition (SVD) is a promising post-training compression technique for large language models (LLMs), as it is training-free, hardware-friendly, and preserves regular dense computation. However, existing SVD-based methods suffer from severe accuracy degradation under aggressive compression. This is mainly because hard rank truncation discards useful singular directions, while the two resulting SVD factors are usually compressed symmetrically despite their different roles, shapes, and quantization sensitivities. In this paper, we propose DELTA, an asymmetric low-rank compression method that decouples the latent heterogeneity between the two SVD factors. Under a fixed memory budget, the proposed method jointly determines the retained rank, factor-wise quantization bit-widths, and quantization directions, allowing more effective singular directions to be preserved with low-bit representations. To efficiently select compression configurations, we use an activation-aware reconstruction error as a layer-wise proxy objective. Experiments on multiple LLMs and datasets show that DELTA consistently outperforms existing post-training SVD-based compression methods, especially under high compression ratios. Compared with the strongest SVD-based baseline at the same compression ratio, DELTA reduces perplexity on WikiText-2 and C4 by up to 68.2% and 53.2%, respectively, and improves average zero-shot accuracy by up to 11 percentage points. These results demonstrate that asymmetric factor-aware compression can effectively mitigate the accuracy degradation of SVD-based LLM compression at high compression ratios. Code is available at https://github.com/BlacksheepJane/DELTA.

Mixture-of-Experts (MoE) architectures scale Large Language Model (LLM) capacity efficiently by activating a sparse subset of experts per token. However, modern MoE inference remains heavily constrained by the rigid, whole-expert abstraction. Existing frameworks manage, schedule, or prune experts as atomic execution units, which fixes the optimization boundary too early and leaves fine-grained intra-expert computational redundancy underexplored. In this work, we present PCoMoE, a path-compositional execution framework that shifts MoE inference from coarse-grained expert selection to fine-grained path composition. PCoMoE incorporates a path-level formulation of expert computation, a compatibility-aware layer-wise pruning strategy to suppress low-value path combinations, and a hardware-friendly execution engine to exploit reusable sub-expert structures under strictly bounded overheads. Experimental results demonstrate that PCoMoE achieves up to a 1.31x end-to-end inference speedup while enhancing model accuracy by 10%. The code is available at https://github.com/gzyyy0/PCoMoE