Current research on LoRA primarily focuses on minimizing the number of fine-tuned parameters or optimizing its architecture. However, the necessity of all fine-tuned LoRA layers during inference remains underexplored. In this paper, we investigate the contribution of each LoRA layer to the model’s ability to predict the ground truth and hypothesize that lower-layer LoRA modules play a more critical role in model reasoning and understanding. To address this, we propose a simple yet effective method to enhance the performance of large language models (LLMs) fine-tuned with LoRA. Specifically, we identify a “boundary layer” that distinguishes essential LoRA layers by analyzing a small set of validation samples. During inference, we drop all LoRA layers beyond this boundary. We evaluate our approach on three strong baselines across four widely-used text generation datasets. Our analysis and experimental results show that selectively retaining critical LoRA layers during inference is effective across tasks of different types.

Large Audio Language Models (LALMs) are increasingly applied in affective and conversational AI, yet their ability to align with human empathetic judgments in spoken dialogue remains poorly understood. We introduce BAEmpEval, a multimodal evaluation framework for analyzing empathetic response judgment in instruction-following LALMs under subjective conversational settings. The framework contains 5,436 dialogue pairs collected from three speech sources, where models evaluate listener responses across six fine-grained dimensions spanning both vocal perception (Vocal Warmth, Perceived Genuineness, Prosodic Attunement) and conversational response strategies (Exploration, Interpretation, Emotional Reaction). Experiments reveal a challenging setting: experiments across recent LALMs reveal uneven alignment with human relational judgments across empathy dimensions and speech conditions. Our findings suggest that empathetic response judgment remains a challenging multidimensional perceptual task for current LALMs and highlight the importance of evaluation frameworks that account for subjective relational variability. We release the dataset and evaluation code to support future research in human-centric AI.

Large Audio Language Models (LALMs) exhibit strong speech understanding ability, yet it remains unclear whether they meaningfully utilize subtle speech-conditioned emotional cues beyond coarse acted emotions. In this work, we investigate whether the evaluated LALMs exhibit measurable sensitivity to fine-grained emotional variation in speech through controlled perturbation experiments, fine-grained emotion elicitation tasks, and representation geometry analysis. Our results show that vocal delivery variations systematically influence both model behavior and hidden representations, indicating that the evaluated LALMs already encode non-trivial speech-conditioned affective information. However, models struggle under emotionally ambiguous settings, particularly for semantically related fine-grained emotions. Representation analysis further suggests that subtle emotional structure is partially preserved despite weak downstream utilization. Overall, our findings suggest that the evaluated LALMs do not completely ignore subtle speech emotion cues, but instead utilize them inconsistently under ambiguous affective conditions