Large language models (LLMs) increasingly provide practical advice in contexts where users’ religious identities are relevant. Existing evaluations of religion in language models have primarily examined stereotypes, representational bias, omission, emotional attribution, or differential treatment across religious groups. We study a different failure mode: whether an LLM preserves the correspondence between a user’s religious identity and the religious practice instantiated in subsequent practical advice. We evaluate 1,260 responses from seven LLMs in controlled fiveturn conversations spanning six religious conditions, six destination settings, and four practical dimensions. All responses were manually annotated using five ordinal quality dimensions, Religious Identity Preservation (RIP), and an eight-label primary error taxonomy. Overall, 437 of 1,260 responses (34.68%) contained an annotated error. Errors were strongly concentrated in the fasting-related Time Representation condition, with an error rate of 49.21% compared with 22.2-27.0% in the other dimensions. A conversation-clustered Generalized Estimating Equations (GEE) model estimated 3.20 times greater odds of an overall error relative to Culture (95% CI [2.24, 4.58], p < .001). We further identify Schema Transfer, in which a response retains the relevant religious identity while instantiating a practice associated with a different religious schema. Schema Transfer occurred in 118 of 1,260 responses (9.37%) and was strongly concentrated in Time Representation, where it occurred in 109 of 504 responses (21.63%), compared with 9 of 756 responses (1.19%) in the remaining dimensions. The pattern was present across both fastingrelated Time turns (23.41% and 19.84%), rather than being driven by a single prompt. An adjusted conversation-clustered GEE estimated 25.25 times greater odds of Schema Transfer in Time Representation (95% CI [12.84, 49.68], p < .001). These findings show that surface-level identity preservation is insufficient for evaluating religion-sensitive LLM behavior: a model may preserve the identity while losing the practice.

Emotion detection is an important text classification task with applications in sentiment analysis, social media monitoring, human-computer interaction, and affective language understanding. However, Punjabi written in the Shahmukhi script remains severely under-resourced for emotion detection, with limited benchmark-style resources available for supervised evaluation. This paper introduces ShahiEmotion, a new Punjabi Shahmukhi emotion detection dataset containing 30379 sentence-level instances annotated with seven emotion categories: sadness, surprise, happiness, anger, neutral, fear, and disgust. The dataset is designed to support research in a low-resource setting characterized by script-specific challenges, lexical variation, and substantial class imbalance. We establish baseline results using several pretrained transformer-based models and formulate emotion detection as a sentence-level classification task. In particular, we fine-tune multilingual BERT, multilingual DistilBERT, XLM-RoBERTa, and Urdu RoBERTa under the same training and evaluation setting using standard cross-entropy loss. Experimental results show that XLM-RoBERTa provides the strongest overall performance among the compared models. The best model achieves 77.95% accuracy, 58.47% macro-F1, and 77.60% weighted-F1 on the test set. The dataset, evaluation protocol, and baseline results introduced in this work are intended to support future research on Punjabi Shahmukhi emotion analysis and low-resource NLP.

Lemmatization is an important preprocessing step in Natural Language Processing (NLP); however, annotated resources for medieval languages such as Old Church Slavonic (OCS) are limited in scope, size, and diversity. This paper presents the annotated resources for OCS lemmatization, including annotation process, design choices and non-standard Unicode related issues. The annotated corpus is used to evaluate existing lemmatization tools (Stanza and UDPipe-2 models trained on the UD 2.12 treebank, and a dictionary-based approach) both in cross-dataset and on a corpus obtained by merging the new annotations with existing UD V2.12 OCS data. Pretrained models perform poorly (≈ 15–16%), below a dictionary baseline (≈ 38%), while retraining on the new data improves performance (up to ≈ 51%) and shows different cross-dataset generalization. Experiments in cross-dataset and on the combined corpus demonstrate that lemmatization performance depends strongly on dataset similarity, annotation conventions, and orthographic mismatch. Overall, the findings show the value of the newly annotated resources and the importance of extending OCS lemmatization benchmarks for historical Slavic NLP.