Although machine translation is often considered solved for high-resource languages, it remains challenging for texts involving poetic language, philosophical concepts, and layered metaphors, characteristics that are central to Sanskrit literature. Additionally, Sanskrit’s rich morphology, sandhi, and compounding, combined with its multi-millennial and multi-domain textual tradition, highlight the need for large, diverse parallel resources, which are currently scarce. We introduce Mitrasaṃgraha, a large-scale Sanskrit-English machine translation dataset containing 391,548 aligned sentence pairs, over 4 times larger than the previously available Itihāsa corpus (Aralikatte et al., 2021). The dataset spans more than 3 millennia of Sanskrit literature across multiple domains and includes temporally annotated data for studying domain and period effects on translation performance. We release manually corrected development (5,587) and test (5,552) sets. Benchmarks show that fine-tuning models such as NLLB and Gemma substantially improves translation quality, and retrieval-augmented prompting further enhances performance. In a controlled cross-dataset comparison, training on Mitrasaṃgraha outperforms both Itihāsa and web-mined bitext for two model families. Despite these gains, our results reveal persistent challenges in translating complex compounds, philosophical terminology, and metaphorical language in Sanskrit, highlighting the need for further research in this area.

Buddhist literature is preserved in a network of classical languages, above all Sanskrit, Pāli, Buddhist Chinese, and Tibetan, for which machine translation support remains limited, and largely restricted to translation into English. In order to address this shortcoming, we present three connected contributions. First, we release MITRA-parallel v2, a corpus of 1.69 million automatically aligned parallel records (2.34 million segment pairs) between Sanskrit, Tibetan, and Buddhist Chinese, mined with an embedding-based span-mining pipeline. Second, we present MITRA-MT, a domain-adapted large language model built by continued pretraining of Qwen3.5-9B on a 22.6-billion-token corpus of classical Asian languages and related modern material, followed by a lightweight translation fine-tune and pref- erence optimization. Third, we introduce a multi-directional evaluation suite covering 17 translation directions at sentence level (ten of them also at paragraph level), including cross-classical directions (Sanskrit↔Tibetan, Sanskrit↔Chinese, Tibetan↔Chinese) and from Tibetan into seven modern languages beyond English. On this suite, MITRA-MT outperforms all open baselines, including instruction-tuned LLMs up to 122B parameters and dedicated MT systems, on all directions, and matches commercial references on the large majority of directions when evaluated via chrF, with the largest margins for translation into Tibetan, Sanskrit, and Classical Chinese. We release the parallel corpus, the evaluation data, and the model weights.

With the advent of large language models, machine translation (MT) has become a widely used, but little understood, tool for accessing historical and multilingual texts. While models like GPT, Claude, and Deepseek increasingly enable translation of low-resource and ancient languages, critical questions remain about their evaluation, optimal model selection, and the value of domain-specific training and retrieval-augmented generation setups.While AI models like GPT, Claude, and Deepseek are improving translation capabilities for low-resource and ancient languages, researchers still face important questions about how to evaluate their performance, which models work best, and whether specialized training approaches provide meaningful improvements in translation quality.This study introduces a comprehensive evaluation dataset for Buddhist Chinese to English translation, comprising 2,662 bilingual data points from 32 texts that have been selected to represent the full breadth of the Chinese Buddhist canon.We evaluate various computational metrics of translation quality (BLEU, chrF, BLEURT, GEMBA) against expert annotations from five domain specialists who rated 182 machine-generated translations. Our analysis reveals that LLM-based GEMBA scoring shows the strongest correlation with human judgment, significantly outperforming traditional metrics. We then benchmark commercial models (GPT-4 Turbo, Claude 3.5, Gemini), open-source models (Gemma 2, Deepseek-r1), and a domain-specialized model (Gemma 2 Mitra) using GEMBA. Our results demonstrate that domain-specific training enables open-weights models to achieve competitive performance with commercial systems, while also showing that retrieval-augmented generation (RAG) significantly improves translation quality for the best performing commercial models.

Morphologically rich languages are notoriously challenging to process for downstream NLP applications. This paper presents a new pretrained language model, ByT5-Sanskrit, designed for NLP applications involving the morphologically rich language Sanskrit. We evaluate ByT5-Sanskrit on established Sanskrit word segmentation tasks, where it outperforms previous data-driven approaches by a considerable margin and matches the performance of the current best lexicon-based model. It is easier to deploy and more robust to data not covered by external linguistic resources. It also achieves new state-of-the-art results in Vedic Sanskrit dependency parsing and OCR post-correction tasks. Additionally, based on the Digital Corpus of Sanskrit, we introduce a novel multitask dataset for the joint training of Sanskrit word segmentation, lemmatization, and morphosyntactic tagging tasks. We fine-tune ByT5-Sanskrit on this dataset, creating a versatile multitask model for various downstream Sanskrit applications. We have used this model in Sanskrit linguistic annotation projects, in information retrieval setups, and as a preprocessing step in a Sanskrit machine translation pipeline. We also show that our approach yields new best scores for lemmatization and dependency parsing of other morphologically rich languages. We thus demonstrate that byte-level pretrained language models can achieve excellent performance for morphologically rich languages, outperforming tokenizer-based models and presenting an important vector of exploration when constructing NLP pipelines for such languages.

Buddhist Classical Chinese is a challenging low-resource language that has not yet received much dedicated attention in NLP research. Standard commercial machine translation software performs poorly on this idiom. In order to address this gap, we present a novel dataset of 209,454 bitext pairs for the training and 2.300 manually curated and corrected bitext pairs for the evaluation of machine translation models. We finetune a number of encoder-decoder models on this dataset and compare their performance against commercial models. We show that our best fine-tuned model outperforms the currently available commercial solutions by a considerable margin while being much more cost-efficient and faster in deployment. This is especially important for digital humanities, where large amounts of data need to be processed efficiently for corpus-level operations such as topic modeling or semantic search. We also show that the commercial chat system GPT4 is surprisingly strong on this task, at times reaching comparable performance to our finetuned model and clearly outperforming standard machine translation providers. We provide a limited case study where we examine the performance of selected different machine translation models on a number of Buddhist Chinese passages in order to demonstrate what level of quality these models reach at the moment.

Having access to high-quality grammatical annotations is important for downstream tasks in NLP as well as for corpus-based research. In this paper, we describe experiments with the Latin BERT word embeddings that were recently be made available by Bamman and Burns (2020). We show that these embeddings produce competitive results in the low-level task morpho-syntactic tagging. In addition, we describe a graph-based dependency parser that is trained with these embeddings and that clearly outperforms various baselines.

This paper presents the development of SansTib, a Sanskrit - Classical Tibetan parallel corpus automatically aligned on sentence-level, and a bilingual sentence embedding model. The corpus has a size of about 317,289 sentence pairs and 14,420,771 tokens and thereby is a considerable improvement over previous resources for these two languages. The data is incorporated into the BuddhaNexus database to make it accessible to a larger audience. It also presents a gold evaluation dataset and assesses the quality of the automatic alignment.

The paper introduces end-to-end neural network models that tokenize Sanskrit by jointly splitting compounds and resolving phonetic merges (Sandhi). Tokenization of Sanskrit depends on local phonetic and distant semantic features that are incorporated using convolutional and recurrent elements. Contrary to most previous systems, our models do not require feature engineering or extern linguistic resources, but operate solely on parallel versions of raw and segmented text. The models discussed in this paper clearly improve over previous approaches to Sanskrit word segmentation. As they are language agnostic, we will demonstrate that they also outperform the state of the art for the related task of German compound splitting.