Lightweight neural network models deployed on edge devices are susceptible to performance degradation under unseen acoustic conditions due to domain mismatch between training and deployment environments. This is particularly relevant in causal low latency applications such as cochlear implants (CIs), where limited temporal context and strict computational constraints restrict model robustness. Domain adversarial learning can improve generalization by encouraging representations that are both task-discriminative and domain-invariant; however, jointly learning these representations becomes difficult in limited capacity models. We introduce an adapter-based adversarial learning framework for robust and parameter-efficient domain adaptation. The proposed approach incorporates an adapter module into a pretrained causal backbone. We first establish a robust pretrained baseline network and subsequently perform adversarial adaptation under mismatched acoustic conditions. We evaluate the proposed framework on framewise phonetic group classification to guide phoneme-based time-frequency mask estimation. Empirical results demonstrate consistent improvements in phoneme classification over both standard supervised training and conventional adversarial learning, particularly under low signal-to-noise ratio levels, resulting in statistically significant improvements in objective intelligibility measures of CI vocoded speech.

Cochlear implant (CI) users often experience reduced speech understanding in noisy and reverberant environments. Our recent work has shown the benefits of exploiting knowledge of speech content to drive speech enhancement for CI applications by estimating time-frequency masks based on predictions from a phoneme classifier. However, phonemes with similar temporal and spectral characteristics are likely to be confused, leading to misclassifications that negatively impact the downstream phoneme-based task. Also, machine learning models are generally vulnerable to performance degradation in unseen acoustic conditions. Using a lightweight and causal phoneme-based speech enhancement model developed for CI applications, we demonstrate the utility of alternative phoneme groups to improve model robustness in diverse acoustic settings. Results from objective measures of speech enhancement show that manner of articulation (MOA) groups provide a balance between fine-grained but error-prone phonemes and limited acoustic specificity with voicing groups. In particular, a counterintuitive finding is that MOA-based models trained in reverberant conditions are particularly robust under mismatched train-test conditions. Our findings highlight the importance of linguistically motivated phonetic grouping and selective training strategies for developing robust speech enhancement models for CI applications while maintaining lightweight model complexity for edge deployment.