Audio-Visual Speech Recognition (AVSR) systems typically consume only the final-layer representation of their modality-specific encoders. In self-supervised (SSL) encoders, lower and upper layers encode acoustic and more abstract linguistic cues respectively, so the most informative layer may differ from token to token. We propose Mixture of Layer-wise Tokens (MoLT), which aggregates the per-layer outputs of the AV-HuBERT encoder per token through an MoE-style router. A sparse top-k MoLT (S-MoLT) selects the most useful layers per token, and Dynamic routing-based MoLT (D-MoLT) replaces the fixed k with a learnable per-token threshold. To align routing with recognition difficulty, we further add a token-difficulty-aware lower-upper preference that biases, rather than overrides, the learned routing: Difficulty-adaptive D-MoLT (DD-MoLT) derives the difficulty signal from one modality, while Cross-modal DD-MoLT (CDD-MoLT) uses the audio-video reliability gap. On LRS3, our models consistently improve WER over a strong MMS-LLaMA baseline across SNR conditions, with the gain widening under severe noise (e.g., 13.85 vs. 17.10% at−15 dB), at negligible overhead.

This paper proposes a deep-learning based native-language identification (NLI) using a latent semantic analysis (LSA) as a participant (ETRI-SLP) of the NLI Shared Task 2017 where the NLI Shared Task 2017 aims to detect the native language of an essay or speech response of a standardized assessment of English proficiency for academic purposes. To this end, we use the six unit forms of a text data such as character 4/5/6-grams and word 1/2/3-grams. For each unit form of text data, we convert it into a count-based vector, extract a 2000-rank LSA feature, and perform a linear discriminant analysis (LDA) based dimension reduction. From the count-based vector or the LSA-LDA feature, we also obtain the output prediction values of a support vector machine (SVM) based classifier, the output prediction values of a deep neural network (DNN) based classifier, and the bottleneck values of a DNN based classifier. In order to incorporate the various kinds of text-based features and a speech-based i-vector feature, we design two DNN based ensemble classifiers for late fusion and early fusion, respectively. From the NLI experiments, the F1 (macro) scores are obtained as 0.8601, 0.8664, and 0.9220 for the essay track, the speech track, and the fusion track, respectively. The proposed method has comparable performance to the top-ranked teams for the speech and fusion tracks, although it has slightly lower performance for the essay track.