Parameter-efficient fine-tuning (PEFT) typically relies on additive weight perturbation. However, in dynamic deployment environments like multi-tenant cloud serving or mobile edge devices, additive methods force a dilemma: merge weights and lose task-switching flexibility, or remain unmerged and suffer severe parallel-branching latency. To break this bottleneck, we propose Latent Task Embedding (LatTE), which shifts the PEFT design axis to geometric expansion. Instead of perturbing frozen parameters, LatTE concatenates a task-specific auxiliary embedding to the input and expands the corresponding weight matrices. This structural fusion ensures the forward pass remains a single, optimized matrix multiplication, bypassing the routing overhead of unmerged adapters. Furthermore, LatTE natively supports router-free, intra-batch multi-tasking by isolating tasks into distinct subspaces controlled via binary masking. Extensive evaluations across 1B to 14B parameter models demonstrate that LatTE matches highly optimized baselines like LoRA, while raising throughput by 1.7× over routed multi-adapter LoRA under heterogeneous-task batching.

Large language models (LLMs) are increasingly deployed with task-specific adapters catering to multiple downstream applications. In such a scenario, the additional compute associated with these apparently insignificant number of adapter parameters (typically less than 1% of the base model) turns out to be disproportionately significant during inference time (up to 2.5x times that of the base model). In this paper, we propose a new zero-latency fused low-rank adapter (zFLoRA) that introduces zero or negligible latency overhead on top of the base model. Experimental results on LLMs of size 1B, 3B and 7B show that zFLoRA compares favorably against the popular supervised fine-tuning benchmarks including low-rank adapters (LoRA) as well as full fine-tuning (FFT). Experiments are conducted on 18 different tasks across three different categories namely commonsense reasoning, math reasoning and summary-dialogue. Latency measurements made on NPU (Samsung Galaxy S25+) as well as GPU (NVIDIA H100) platforms show that the proposed zFLoRA adapters introduce zero to negligible latency overhead.

Since language model pretraining to learn contextualized word representations has been proposed, pretrained language models have made success in many natural language processing tasks. That is because it is helpful to use individual contextualized representations of self-attention layers as to initialize parameters for downstream tasks. Yet, unfortunately, use of pretrained language models for emotion recognition in conversations has not been studied enough. We firstly use ELECTRA which is a state-of-the-art pretrained language model and validate the performance on emotion recognition in conversations. Furthermore, we propose contextual augmentation of pretrained language models for emotion recognition in conversations, which is to consider not only previous utterances, but also conversation-related information such as speakers, speech acts and topics. We classify information based on what the information is related to, and propose position of words corresponding to the information in the entire input sequence. To validate the proposed method, we conduct experiments on the DailyDialog dataset which contains abundant annotated information of conversations. The experiments show that the proposed method achieves state-of-the-art F1 scores on the dataset and significantly improves the performance.