Mixture-of-Experts (MoE) architectures have become a dominant paradigm for scaling large language models (LLMs), but their large memory footprint still makes post-training quantization (PTQ) essential for practical deployment. Unlike dense LLMs, quantized MoE models exhibit routing perturbations, where quantization alters expert routing behavior. Existing router alignment methods mitigate this issue by uniformly minimizing routing discrepancies across all tokens. However, we observe that routing perturbations exhibit highly heterogeneous relationships with observable output discrepancy: many tokens preserve nearly identical output distributions despite substantial routing changes. Motivated by this observation, we propose OPERA, an output-aware router alignment strategy for quantized MoE models that selectively prioritizes tokens exhibiting observable output discrepancy after quantization. We further analyze output-preserving routing perturbations and show that they frequently occur on highly predictable tokens, where substantial routing perturbations and hidden-state deviations may not strongly affect projected output distributions. Experiments across multiple MoE LLMs demonstrate that OPERA generally improves over conventional router alignment, with particularly noticeable gains under aggressive low-bit quantization settings.

Mixture-of-Experts (MoE) language models scale model capacity through sparse expert activation, but their large parameter footprint remains costly to deploy. Post-training quantization (PTQ) offers a practical compression strategy; however, quantization errors can alter router decisions and cause the quantized model to activate experts different from those selected by its full-precision counterpart. Prior router-aware PTQ objectives mitigate such expert-shift by aligning router outputs or preserving top-k expert orderings and margins at a given router. Yet sequential block-wise PTQ introduces an additional failure mode: quantizing a decoder block changes the representations received by the next MoE router, potentially causing downstream expert-shift that is not directly optimized by existing objectives. To address this issue, we propose MENDS-MoE (Margin-Preserving Expert Orderings with Next-Router Decision Supervision), an auxiliary calibration objective that supervises current-block quantization using routing decisions at the next MoE router. MENDS-MoE further introduces a boundary-aware margin loss that preserves orderings among selected experts while maintaining their separation from unselected experts near the top-k boundary. It is applied only during calibration and adds no inference-time computation. Across three MoE language models under 4-bit and 3-bit-average weight-only quantization, MENDS-MoE achieves the best average downstream accuracy and the lowest perplexity in most evaluated model–bit settings. Ablation studies further support the contributions of both next-router supervision and boundary-aware margin preservation.

Recently, large language models (LLMs) have demonstrated unprecedented capabilities in language generation, yet they still often produce incorrect information. Therefore, determining whether a text was generated by an LLM has become one of the factors that must be considered when evaluating its reliability. In this paper, we discuss methods to determine whether texts written in various languages were authored by humans or generated by LLMs. We have discovered that the classification accuracy significantly decreases for texts written in languages not observed during the training process, and we aim to address this issue. We propose a method to improve performance for unseen languages by using token-level predictive distributions extracted from various LLMs and text embeddings from a multilingual pre-trained langauge model. With the proposed method, we achieved third place out of 25 teams in Subtask B (binary multilingual machine-generated text detection) of Shared Task 1, with an F1 macro score of 0.7532.

Recently, large language models (LLMs) have shown remarkable performance across various natural language processing tasks, thanks to their vast amount of knowledge. Nevertheless, they often generate unreliable responses. A common example is providing a single biased answer to an ambiguous question that could have multiple correct answers. To address this issue, in this study, we discuss methods to detect such ambiguous samples. More specifically, we propose a classifier that uses a representation from an intermediate layer of the LLM as input. This is based on observations from previous research that representations of ambiguous samples in intermediate layers are closer to those of relevant label samples in the embedding space, but not necessarily in higher layers. The experimental results demonstrate that using representations from intermediate layers detects ambiguous input prompts more effectively than using representations from the final layer. Furthermore, in this study, we propose a method to train such classifiers without ambiguity labels, as most datasets lack labels regarding the ambiguity of samples, and evaluate its effectiveness.

Question-Answering (QA) has seen significant advances recently, achieving near human-level performance over some benchmarks. However, these advances focus on high-resourced languages such as English, while the task remains unexplored for most other languages, mainly due to the lack of annotated datasets. This work presents a native QA dataset for an East African language, Tigrinya. The dataset contains 10.6K question-answer pairs spanning 572 paragraphs extracted from 290 news articles on various topics. The dataset construction method is discussed, which is applicable to constructing similar resources for related languages. We present comprehensive experiments and analyses of several resource-efficient approaches to QA, including monolingual, cross-lingual, and multilingual setups, along with comparisons against machine-translated silver data. Our strong baseline models reach 76% in the F1 score, while the estimated human performance is 92%, indicating that the benchmark presents a good challenge for future work. We make the dataset, models, and leaderboard publicly available.

Natural language understanding (NLU) tasks face a non-trivial amount of ambiguous samples where veracity of their labels is debatable among annotators. NLU models should thus account for such ambiguity, but they approximate the human opinion distributions quite poorly and tend to produce over-confident predictions. To address this problem, we must consider how to exactly capture the degree of relationship between each sample and its candidate classes. In this work, we propose a novel method with deep model compression and show how such relationship can be accounted for. We see that more reasonably represented relationships can be discovered in the lower layers and that validation accuracies are converging at these layers, which naturally leads to layer pruning. We also see that distilling the relationship knowledge from a lower layer helps models produce better distribution. Experimental results demonstrate that our method makes substantial improvement on quantifying ambiguity without gold distribution labels. As positive side-effects, our method is found to reduce the model size significantly and improve latency, both attractive aspects of NLU products.