LLM agents that repair failures often discard successful corrections, forcing later episodes to rediscover similar solutions. We study whether finalized repair outcomes can improve subsequent Text-to-SQL episodes without parameter updates. We introduce MERIT, a training-free agent that maintains an online dual-polarity memory of oracle-verified corrections and observed unsuccessful directions. Under oracle-assisted benchmark feedback, only memories from earlier finalized episodes are eligible for retrieval. A deterministic classifier assigns a coarse failure type, which conditions a hybrid lexical-dense retriever before the frozen model generates each revision. Using Qwen2.5-7B-Instruct with identical initial predictions and repair budgets, MERIT improves execution accuracy over stateless iterative repair from 66.34% to 69.79% on Spider and from 47.35% to 48.44% on BIRD. Paired analyses provide clear evidence for the Spider gain but weaker evidence on BIRD. MERIT is not reliably separated from untyped dynamic retrieval on either benchmark, while Reflexion-style memory reaches 51.24% on BIRD at substantially higher inference cost. Ablations show that negative memory contributes modestly, the value of type conditioning and lexical-dense ranking is dataset dependent, and schema-local experience provides the most consistent benefit. These results clarify when causal cross-query memory improves repair and when broader memory representations remain preferable. Our implementation is available here: https://github.com/nhatkhangcs/merit

Open Domain Multi-hop Question Answering faces a dual compositionality challenge: reasoning over complex query structures and integrating evidence scattered across contexts. Despite recent advancements in Graph-based Retrieval-Augmented Generation (GraphRAG), persistent limitations in complex reasoning and retrieval inaccuracies continue to constrain the efficacy of multi-hop QA systems. We introduce HiGraAgent, a framework that unifies graph-based retrieval with adaptive reasoning. It constructs a Hierarchical Knowledge Graph (HiGra) with entity alignment, reducing redundancy by 34.5% while preserving expressiveness; employs HiGraRetriever, a hybrid graph-semantic retriever that consistently outperforms the strongest graph-based method across benchmarks; and integrates a dual-agent adaptive reasoning protocol where a Seeker and a Librarian dynamically coordinate retrieval and reasoning. Together, these innovations enable HiGraAgent to achieve 85.3% average accuracy on HotpotQA, 2WikiMultihopQA, and MuSiQue, surpassing the strongest prior system by 11.7%. Our results highlight the importance of reframing multi-hop QA as a problem of adaptive reasoning, offering a more robust and flexible paradigm for complex information seeking.

Large Language Models (LLMs) depend on retrieval for factual grounding in Retrieval-Augmented Generation (RAG), placing Information Retrieval (IR) at the core of modern Question Answering (QA) systems. While lexical, dense, and hybrid paradigms have been extensively benchmarked in English, their relative effectiveness for Vietnamese remains insufficiently characterized, especially under realistic multi-domain settings. Existing studies are typically confined to single domains or curated datasets, limiting cross-domain comparability and obscuring paradigm-level trade-offs. We introduce the first domain-normalized, multi-domain benchmark for Vietnamese IR under a unified and reproducible evaluation protocol, spanning six domains and ten datasets across education, legal, healthcare, customer support, lifestyle reviews, and open-domain knowledge. We evaluate lexical, neural-sparse, late-interaction, dense, and hybrid paradigms across diverse Vietnamese-specific and multilingual embedding backbones, and release two QA datasets, EduCoQA and CSConDa, constructed from authentic counseling and customer-service interactions. Beyond reporting benchmark performance, we derive systematic insights into lexical–semantic hybridization, specialization versus robustness trade-offs, and the limited predictive value of model scale for retrieval effectiveness. All datasets and evaluation scripts are publicly available at https://github.com/longstnguyen/ViRE.

Large language models (LLMs) exhibit strong English performance due to its dominance in pretraining data, yet remain inconsistent across non-English languages. Existing cross-lingual alignment methods rely on contrastive objectives that symmetrically perturb English representations, inadvertently degrading the proficiency they use as a reference. We propose an Optimal Transport (OT) framework that maps non-English hidden-state distributions toward English at middle transformer layers via entropy-regularised Sinkhorn iterations, while an English consistency regularisation term keeps the alignment anchor stationary. This asymmetric formulation preserves English performance by construction. Experiments on XNLI, XQuAD, and MMMLU show that our method delivers strong cross-lingual transfer while successfully avoiding the English performance degradation observed in the contrastive baseline.

Large language models are effective at code completion when the required evidence is local, but repository-level fill-in-the-middle completion remains challenging when it depends on code and structural relationships across files. We present ACAR, a framework that combines lexical and AST/LSP-based structural retrieval through line-level reciprocal rank fusion, then applies syntax-aware post-processing to align generated completions with the surrounding code. We evaluate ACAR on RepoEval, ReccEval, and CrossCodeEval across Python, Java, TypeScript, and C#, covering more than 3,000 real-world repositories. In the main Qwen2.5-Coder experiments, ACAR outperforms strong retrieval baselines in most settings, reaching 42.98 EM on ReccEval and 32.86 on CrossCodeEval at 3B, versus 36.25 and 28.40 for the strongest retrieval baseline, respectively. With the largest tested Qwen and DeepSeek backbones, ACAR improves EM over their corresponding no-retrieval baselines across all five evaluation settings. Ablations show that lexical and structural evidence are complementary, while removing syntax-aware refinement causes the largest drop in edit similarity. These gains come with 87–95 ms of non-decoding overhead under warm-cache conditions across the three benchmarks, supporting latency-sensitive code completion. To support reproducibility, we make our implementation and evaluation scripts publicly available at https://github.com/longstnguyen/ACAR.

Mixture-of-Experts (MoE) routing collapses in parameter-efficient fine-tuning: on LLaMA-3 8B with MixLoRA, a single expert absorbs 82.5% of tokens. Prior MoE-PEFT remedies intervene at the output: routing reformulations, output-similarity penalties, or shared-expert capacity. All leave each routed expert’s input view identical. We propose SIMPA (Sparse-Input Magnitude Partitioning Adaptation), which restricts what experts *see* rather than what they produce: a magnitude-thresholded input drives a sparse routed-LoRA branch, a parallel dense LoRA branch (adapting DeepSeekMoE’s (Dai et al., 2024) shared-expert topology to PEFT) provides an information safeguard, and a column-wise gate fuses the two. At a matched 2.9% trainable-parameter budget, SIMPA leads compared baselines on reasoning (86.2% on seven LLaMA-3 8B tasks, +3.9 over MixDoRA), code and arithmetic (38.4 HumanEval Pass@1 and 59.4 GSM8K, matching full fine-tuning under identical data and compute), and NLU (88.7 GLUE / 77.7 SuperGLUE on T5-base); against direct MoE-PEFT counterparts HydraLoRA and MoLoRA at matched setup, SIMPA leads HumanEval by +14.2 and +13.6 and GSM8K by +2.1 and +2.8. Direct measurement returns an expert-input Jaccard of 0.184 versus MixLoRA’s 1.000, a 5.4× reduction. The same block-aligned mask drives a Triton kernel measuring 1.46× end-to-end inference speedup on A100. Closed-form partitioning and retention bounds characterize the mechanism, and a tail-index-driven closed form predicts the optimal sparsity p^* from the pretrained backbone. The mechanism, its predicted optimum, and its bound all transfer to a held-out Qwen2.5-7B backbone. Code: https://github.com/triquang26/simpa.

In isolating languages such as Vietnamese, core morphological structure is encoded not by inflection but by the composition and ordering of monosyllabic morphemes, yet standard Transformer encoders largely overlook this signal. We introduce HuTieuBERT, a morpheme-aware Transformer that augments a pretrained Vietnamese encoder with two lightweight inductive biases: (i) Adaptive Boundary-Token Fusion, which integrates BMES-based morpheme boundary embeddings into token representations via a learnable gate, and (ii) a Morpheme-Aware Attention Bias, which injects a fixed structural attention matrix into early self-attention layers while minimally perturbing the pretrained attention geometry. Across a suite of Vietnamese POS, NER, and sentence-level classification benchmarks, HuTieuBERT consistently outperforms strong baselines, with the largest gains on syntactic tasks. Hyperparameter ablations show a broad regime in which structural biases improve accuracy without destabilizing representations. Applying the same design to ChineseBERT (Chinese-BERT-wwm) yields MAChineseBERT, which improves F₁ and produces more balanced tag distributions on Chinese POS and NER, suggesting that explicit morpheme-aware attention is a portable and effective strategy for modeling isolating languages.

The Bahnar people, one of Vietnam’s ethnic minorities, represent an underserved community with limited access to modern technologies. Developing an effective Bahnaric-Vietnamese translation system is essential for fostering linguistic exchange, preserving cultural heritage, and empowering local communities by bridging communication barriers. With advancements in Artificial Intelligence (AI), Neural Machine Translation (NMT) has achieved remarkable success across various language pairs. However, the low-resource nature of Bahnaric, characterized by data scarcity, vocabulary constraints, and the lack of parallel corpora, poses significant challenges to building an accurate and efficient translation system. To address these challenges, we propose a novel hybrid architecture for Bahnaric-Vietnamese translation, with BARTBahnar as its core language model. BARTBahnar is developed by continually training a pre-trained Vietnamese model, BARTPho, on augmented monolingual Bahnaric data, followed by fine-tuning on bilingual datasets. This transfer learning approach reduces training costs while effectively capturing linguistic similarities between the two languages. Additionally, we implement advanced data augmentation techniques to enrich and diversify training data, further enhancing BARTBahnar’s robustness and translation accuracy. Beyond leveraging the language model, our hybrid system integrates rule-based and statistical methods to improve translation quality. Experimental results show substantial improvements on bilingual Bahnaric-Vietnamese datasets, validating the effectiveness of our approach for low-resource translation. To support further research, we open-source our code and related materials at https://github.com/ura-hcmut/BARTBahnar.

Large Language Models (LLMs) have shown strong capabilities in Question Answering (QA), but their effectiveness in high-stakes, closed-domain settings is often constrained by hallucinations and limited handling of vague or underspecified queries. These challenges are especially pronounced in Vietnamese, a low-resource language with complex syntax and strong contextual dependence, where user questions are often short, informal, and ambiguous. We introduce the Unified Retrieval Agent-Based System (URASys), a QA framework that combines agent-based reasoning with dual retrieval under the Just Enough principle to address standard, ambiguous, and unanswerable questions in a unified manner. URASys performs lightweight query decomposition and integrates document retrieval with a question–answer layer via a two-phase indexing pipeline, engaging in interactive clarification when intent is uncertain and explicitly signaling unanswerable cases to avoid hallucination. We evaluate URASys on Vietnamese and English QA benchmarks spanning single-hop, multi-hop, and real-world academic advising tasks, and release new dual-language ambiguous subsets for benchmarking interactive clarification. Results show that URASys outperforms strong retrieval-based baselines in factual accuracy, improves unanswerable handling, and achieves statistically significant gains in human evaluations for clarity and trustworthiness. All code and datasets are publicly available at https://github.com/ura-hcmut/URASys.

Recent advancements in large language models (LLMs) have underscored their importance in the evolution of artificial intelligence. However, despite extensive pretraining on multilingual datasets, available open-sourced LLMs exhibit limited effectiveness in processing Vietnamese. The challenge is exacerbated by the absence of systematic benchmark datasets and metrics tailored for Vietnamese LLM evaluation. To mitigate these issues, we have finetuned LLMs specifically for Vietnamese and developed a comprehensive evaluation framework encompassing 10 tasks and 31 metrics. We observe that finetuning can help LLMs transfer knowledge across languages, serving as an efficient way to bolster their capabilities in non-English languages. Moreover, our analysis indicates that larger models can introduce more biases and uncalibrated outputs and the key factor influencing LLM performance is the quality of the training or finetuning datasets. These insights underscore the significance of meticulous finetuning with high-quality datasets in enhancing LLM performance.

Recently, Transformer-based models have been proven effective in the abstractive summarization task by creating fluent and informative summaries. Nevertheless, these models still suffer from the short-range dependency problem, causing them to produce summaries that miss the key points of document. In this paper, we attempt to address this issue by introducing a neural topic model empowered with normalizing flow to capture the global semantics of the document, which are then integrated into the summarization model. In addition, to avoid the overwhelming effect of global semantics on contextualized representation, we introduce a mechanism to control the amount of global semantics supplied to the text generation module. Our method outperforms state-of-the-art summarization models on five common text summarization datasets, namely CNN/DailyMail, XSum, Reddit TIFU, arXiv, and PubMed.