Tan Sang Nguyen

Author directory

2026

Large language models (LLMs) have proven to be effective tools for a wide range of natural language processing (NLP) applications. Although many LLMs are multilingual, most remain English-centric and perform poorly on low-resource languages. Recently, several Southeast Asia–focused LLMs have been developed, but none are truly open source, as they do not publicly disclose their training data. Truly open-source models are important for transparency and for enabling a deeper and more precise understanding of LLM internals and development, including biases, generalization, and multilinguality. Motivated by recent advances demonstrating the effectiveness of parallel data in improving multilingual performance, we conduct controlled and comprehensive experiments to study the effectiveness of parallel data in continual pretraining of LLMs. Our findings show that using only parallel data is the most effective way to extend an LLM to new languages. Using just 34.7B tokens of parallel data and 180 hours on 8× NVIDIA H200 GPUs, we built OpenSeal, the first truly open Southeast Asian LLM that rivals the performance of existing models of similar size.
For very low-resource language pairs such as Bahnar-Vietnamese, the relative effectiveness of generic multilingual, language-pair-specific lexical, and task-adapted retrieval remains unclear. We study Bahnar-Vietnamese parallel-sentence retrieval (bitext retrieval) as a controlled candidate-ranking task and compare lexical, alignment-based, neural, and hybrid methods under a development-based selection protocol. The strongest off-the-shelf encoder, LaBSE, reaches 24.24% Recall@1, while symmetric IBM1 reaches 49.33%. Among four in-domain modern dense retrievers, development selection chooses DPR-XM, which reaches 47.43% Recall@1 and 72.21% Recall@10. XLM-R LoRA reaches 64.02% Recall@10, and the development-selected XLM-R-first IBM1 reranker gives the highest Recall@1 point estimate, at 51.62%, with an MRR of 0.5579. Paired tests do not establish a significant difference between the hybrid and IBM1 at the 0.05 level, although reranking substantially improves over its neural first stage. Under pair-specific training, LoRA projection with CSLS also performs strongly on Khmer-Vietnamese and Lao-Vietnamese, reaching 80.14% and 84.58% Recall@1. Overall, the results show complementary strengths between lexical alignment and lightweight neural adaptation in low-resource bitext retrieval. Data and code are available at https://github.com/NgThTy/translation-vn-bahna.