Bengali is spoken by more than 230 million people, yet no standardized instrument evaluates large language models (LLMs) on Bengali across the task categories used to benchmark frontier models. We release 8 English benchmarks translated into Bengali with a single consistent pipeline and use them to evaluate 10 open LLMs from 4 families on paired English and Bengali inputs. Script fragmentation is what subword tokenizers do to Bengali’s alphasyllabary, whose written units are grapheme clusters spanning several Unicode code points: they cut the script into pieces smaller than a character, at a cost set by the vocabulary rather than the script itself. Format belongs to the evaluation, the answer shape that exact-match scoring demands regardless of whether the model knew the answer. Beyond confirming a substantial gap (macro LLM-judge score 0.79 in English versus 0.63 in Bengali), we show that part of it is a measurement artifact: exact-match accuracy conflates correctness with format adherence and because format failure is asymmetric across languages it distorts the apparent gap for some models three to five fold and even reverses its sign for one reasoning-tuned model. On the fragmentation side, Bengali costs roughly five times more tokens per word than English, the 10 models share only 4 vocabularies and that cost varies twofold across them (7.9 tokens per word under Llama3̃, 4.0 under Tekken), and under three of the four the average Bengali token spans fewer bytes than a single code point. Bengali is the constant here and the vocabularies are not, so what Bengali text costs is set by tokenizer design rather than by the script. Fertility and sequence length correlate only weakly with scores (r = -0.23), so we present this as a cost and segmentation concern rather than a driver of the gap. The datasets, pipeline and evaluation code are released.

Speech–text retrieval is a multimodal alignment task in which spoken audio and written transcripts are mapped into a shared representation space. Standard retrieval metrics test whether an audio embedding recovers its transcript from a large candidate pool, but not whether the shared representation preserves small grammatical contrasts inside agglutinated words. We construct an automatic Malayalam benchmark with 12,611 controlled audio–text comparison rows derived from 6,611 held-out audio queries. The benchmark contains three morphosyntactic categories—case, tense, and negation—and a lexical numeral-control category. Alternatives are generated with the mlmorph finitestate analyser and generator and retained after a generate–reanalyse consistency check. A CLIP-style model without explicit hard negatives obtains 37.45% full-pool Recall@1 but only 53.66% pairwise accuracy, close to the 50% pairwise baseline. We diagnose a fixedfirst-candidate loader defect that under-delivers later-generated categories and replace it with per-transcript category-aware, access-time resampling. At the matched epoch count, pairwise accuracy rises from 73.50% to 79.76%, and tense accuracy rises from 52.83% to 77.70%. The complete corrected procedure reaches 89.69% pairwise accuracy after extended training, while full-pool retrieval remains below the original checkpoint. A stratified human review of 797 rows retains or corrects 731 (91.7%); the final checkpoint obtains 74.01% on this deliberately reweighted and partly rewritten subset. These results show that global retrieval, controlled grammatical discrimination, candidate-delivery logic, and human benchmark validation are distinct components of multimodal evaluation.

The manual creation of high-quality study materials such as summaries, question–answer sets, and flashcards remains a time-consuming task for both educators and learners. While large language models can generate such content rapidly, they frequently suffer from hallucinations and lack grounding in the source material. This paper presents a Retrieval-Augmented Generation (RAG) framework designed to address these limitations by transforming educational documents into pedagogically structured learning resources. The system integrates document preprocessing, semantic chunking, embedding-based retrieval, and a locally deployed language model to produce concise summaries, targeted questions, and flashcards. A cross-sectional evaluation involving 48 undergraduate students and 12 educators was conducted to assess usability, accuracy, cognitive support, and overall pedagogical value. Findings indicate that the AI Study Agent is perceived as supportive of comprehension and revision activities, and objective evaluation supports factual consistency relative to non-retrieval generation. Challenges related to output accuracy, oversimplification, and user trust remain consistent with limitations identified in existing GenAI literature. The study concludes that while the system demonstrates strong potential to support teaching and learning workflows, its perceived effectiveness relies on responsible use, educator oversight, and continued refinement to ensure pedagogical alignment and reliability.

Speech sentiment systems are routinely benchmarked with gold transcripts, but a deployed cascade must read whatever its recognizer produces. We audit that gap on BanglaMUSE, a Bangla read-speech sentiment corpus, using sentence-grouped cross-validation and a controlled substitution design in which one trained classifier is evaluated three times while only the test-time transcript changes. Replacing gold transcripts with ASR output costs 0.109 macro-F1 with Bengali XLS-R CTC and 0.341 with our Whisper large-v3 configuration, a high-error stress case. Retraining the classifier on matched recognizer output recovers 57–81% of the gap, but re-selecting only the decision threshold on ASR validation data, with weights and probabilities unchanged, already recovers 28–49%. Adding the speech embedding improves over ASR text, yet replacing it with acquisition/nuisance features gives a similar gain for gold-trained heads. These features alone predict sentiment at 0.669 macro-F1 (group-permutation null mean 0.495, p = 0.001), and a speech-only classifier that reaches 0.940 falls to 0.621 when both speaker and sentence are unseen. These speech-only and fusion results therefore do not provide clean evidence of prosodic sentiment cues. Low-resource speech–text benchmarks should report oracle and deployable conditions, separate threshold effects from retraining, and test acquisition variables before interpreting audio.

Multimodal content moderation in online social spaces is a critical safety challenge, yet existing systems are heavily dominated by monolithic cultural biases and perform poorly on underrepresented languages. In this paper, we study culturally grounded misogynistic meme detection under a novel multi-perspective evaluation framework, which simultaneously assesses models across Indian, Chinese, and Western cultural lenses. We propose MoEzhi-Guru (drawing on the Dravidian term mozhi for language and the Indic term guru for expert/guide), an adaptable, resource-efficient Multimodal Mixture of Experts (MoE) model designed to handle complex cross-cultural and cross-lingual shifts. The system encodes input using mDeBERTa-v3 and SigLIP to extract rich multilingual text and global visual semantics, then fuses and routes these features through a custom MoE layer utilizing dense soft routing. This gating mechanism dynamically weights specialized expert subnetworks, capturing localized socio-cultural nuances before predicting labels through culture-specific classification heads. We evaluate our framework in four typologically diverse language subsets: Tamil and Malayalam (underrepresented Dravidian scripts), Chinese (containing colloquial and code-switched text), and English. Our system demonstrates robust cross-lingual generalization, yielding an overall average score of 0.8515 Macro-F1 and consistently exceeding a Gemma3-4b baseline by more than 0.42 Macro-F1 points. This work provides a scalable, inclusive foundation for culturally aware safety systems in low-resource and multilingual environments.

We present our system for the MMLow@AACL-IJCNLP 2026 shared task on multi-level classification of Tamil political memes. Our approach fine-tunes Qwen2-VL-2B-Instruct with QLoRA under a cascaded prompting strategy, in which a Level 1 stance prediction (Support/Praise vs. Troll/Oppose) conditions the prompt used for Level 2 target classification (party vs. person, per stance). On the official test set, our system achieves a Level 1 macro-F1 of 0.7173 and a Level 2 macro-F1 of 0.4429, for a final score of 0.5801, placing 5th of 7 teams. Our error analysis shows that all Level 1 errors propagate downstream, accounting for the majority of Level 2 mistakes. Additionally, a preliminary qualitative review suggests that sarcasm interpreted literally drives a large share of Level 1 misclassifications, pointing to sarcasm detection and cross-modal reasoning as key bottlenecks for this task.

We describe the TVKS submission to the MMLow 2026 shared task on multilevel classification of Tamil political memes, which asks systems to predict a stance label (Level 1: Support/Praise vs. Troll/Oppose) and a four-way target label (Level 2) that combines stance with a Person/Party distinction. We factorize the Level 2 label into two binary axes—stance and person/party target—reconstructing Level 2 as their product rather than predicting four classes directly. A flat four-way head drives the rarest class (Troll/Oppose Against Party) to an F1 of 0.000; factorization recovers a non-degenerate prediction over all four classes. We also fix a calibration failure caused by stacking a class-balanced sampler with a class-weighted loss, and combine two multimodal models (MuRIL + ViT and MuRIL + ResNet-50) in an axis-wise ensemble that routes each binary axis to its stronger model. Our system reaches a macro-F1 of 0.8594 on Level 1 and 0.5926 on Level 2 (mean 0.7260), ranking second of seven teams. An error analysis attributes 77% of Level 2 errors to the person/party axis, where architecturally distinct visual encoders converge to the same ceiling—evidence that the residual gap is dataset ambiguity rather than a modelling deficiency.

Misogyny and hate content have become a serious issue in the current digital era. Such content predominantly manifests as short video clips, making automated detection mechanisms imperative for social media platforms. To effectively classify such videos, a multimodal setup is required, as video data contains image frames, audio clips, and text captions. The multimodal misogyny video classification task aims to combine all audio, visual, and extracted text information. Conventional methods typically aim to process this extracted information in parallel to reach a conclusion. Our proposed framework utilizes the pre-trained models DINOv2, wav2vec 2.0, and MuRiL-large to embed visual, audio, and text features, respectively. To mitigate overfitting on datasets of different sizes, we propose a dual-architecture approach: we utilize a regularized Gated Early-Fusion Network (EF-Net) for the highly constrained Tamil dataset, and a deeper Cross-Modal Temporal Attention Network (TA-Net) with an ensemble strategy for the Malayalam dataset. Ultimately, our framework achieved a 68.8% macro-F1 score for the Malayalam dataset and an 80.7% macro-F1 score for the Tamil dataset, securing Rank 1 in misogyny video classification for both languages.

Political memes are inherently multimodal, combining visual cues, embedded text, and code-mixed language to express political stance and target specific political entities. This makes automated analysis particularly challenging in low-resource settings such as Tamil. We present the KEC system for the MMLow 2026 shared task at AACL-IJCNLP 2026, which requires hierarchical classification of Tamil political memes at two levels: stance identification and political target identification. Our approach combines XLM-RoBERTa and Vision Transformer representations with OCR-derived textual information, multimodal fusion, and bidirectional cross-modal attention. We further introduce hierarchy-aware prediction in which Level-2 target identification is conditioned on the Level-1 stance distribution, together with class-weighted objectives, adaptive task weighting, and hierarchical consistency regularization. A controlled six-stage ablation study and three-seed stability analysis are conducted on the validation split. The final hierarchical model obtains a Level-1 macro-F1 of 78.77 ± 1.21% and Level-2 macro-F1 of 46.07 ± 3.23%, with a joint exact-match accuracy of 55.40 ± 2.34% across seeds 42, 123, and 2024. Our analysis indicates that visual information is particularly beneficial for stance classification, whereas multimodal fusion plays an important role in target identification, while OCR augmentation and hierarchical conditioning introduce performance trade-offs at Level 2.

Political memes communicate opinions using the interplay between visual and textual information, posing a challenging multimodal classification task. We present a hierarchical multimodal framework for political meme classification, which decomposes the task into stance and target entity prediction. Our framework was evaluated on the data from the Multi-Level Political Meme Classification Shared Task at MMLow @ AACL-IJCNLP 2026. The task comprised two levels of classification: (i) predicting the political stance of a meme as either Support/Praise or Troll/Oppose, and (ii) identifying the target of the meme combined with the stance. We approached the task by first predicting the Level-1 labels followed by predicting the target entity (Person/Party) and then combining them in a deterministic manner to predict the final Level-2 labels. Our method used a SigLIP vision transformer for stance classification and a hybrid SigLIP-XLM-RoBERTa architecture for target classification with late fusion. We trained our models using 5-fold stratified cross-validation with checkpoint ensembling. We also performed ablation experiments to analyze the contribution of individual modalities, observing that visual features are more informative for stance prediction while textual features contain more information for target identification, which supports the multimodal task at hand. Our system achieved a macro F1 score of 0.9027 and 0.6694 for Level-1 and Level-2 classification, respectively, achieving a final score of 0.7860, ranking first in the shared task.

Political memes have become an effective tool for online political communication and engagement. Their interpretation is particularly challenging in low-resource and code-mixed languages such as Tamil, where memes often include Tamil-English, use culturally relevant lexicon, incorporate political entities, and utilize references to prominent personalities and parties. This paper presents our submission to the MMLow shared task at AACL-IJCNLP 2026 on multi-level classification of Tamil political memes, which requires jointly predicting (i) a coarse-grained stance label (Support vs. Troll) and (ii) a fine-grained target label identifying whether the meme concerns a political party or a person. We proposed a lightweight hierarchical classifier built on frozen, pretrained multimodal encoders: images are represented using CLIP (ViT-B/32) and text using the multilingual BGE-M3 embedding model, with the two modalities combined via early, non-parametric concatenation fusion. A shared encoder feeds two prediction heads, where the target-identification head is explicitly conditioned on the stance head’s output logits via a gradient-detachment mechanism, allowing the model to respect the task’s label hierarchy without corrupting the coarse-level classifier’s gradients. To address pronounced class imbalance in both label levels, we applied balanced loss weighting at each level. On the test set of unlabeled data, our system achieved 3rd place, with macro-F1 scores of 0.856 for stance classification and 0.532 for target identification, yielding a final score of 0.694, indicating promising performance on the Tamil political meme classification problem.