Md Adnan Arefeen
Author directory2026
CultAct-ML: Measuring Cultural Value Defaults in Multilingual LLM Agents Under Action
Ahmad Fahmid | Jannatul Ferdous Mim | Md Adnan Arefeen
The Proceedings of the Workshop on Open Reasoning Across Cultures Languages (ORACLE)
Ahmad Fahmid | Jannatul Ferdous Mim | Md Adnan Arefeen
The Proceedings of the Workshop on Open Reasoning Across Cultures Languages (ORACLE)
Measuring which cultural values a languagemodel agent falls back on when it must act is a fundamental problem in multilingual and culturally-grounded evaluation. Current agent benchmarks are designed primarily to measure task success in high-resource languages such as English, which creates major limitations when the question is whose culture an agent’s committed action encodes. The key challenges are the absence of an acting-based value-conflict benchmark and the difficulty of preserving culturallyloaded meaning when a scenario is carried into a low-resource language such as Bangla. In this paper, to study these cultural defaults under action, we present CultAct-ML, a benchmark of 39 value-conflict dilemmas recast as secondperson agent actions and evaluated with a twophase plan–act agent over English, Indonesian, and Bangla. We further introduce three metrics that measure the Western-cluster default, its reliability across languages, and the drift of the agent’s plan, and every Bangla item is verified by a native speaker. Across three models and three repeats, a total of 1,053 plan–act decisions are produced, all valid. We find that a Western default holds above chance for Qwen2.5-72B and Kimi K2 in every language but fades to chance for Qwen3-235B outside English. After stratifying the plan drift by the language of the plan, we further find that Qwen3-235B plans in English rather than Bangla on 70.1% of Bangla scenarios, highlighting a cross-language effect that a pooled analysis hides entirely. The scenarios, code, and all metrics are publicly released.1
Do Small Models Use the Law You Give Them? Measuring Context Use on a Bilingual Bangladesh Legal Benchmark
Moniruzzaman Mahadi | Abrar Mohammed Tanzim Alam | Mir Mohammad Asif Abdullah | Sayma Siddika Monalisa | Swakkhar Shatabda | Md Adnan Arefeen
Proceedings of the Natural Legal Language Processing Workshop 2026
Moniruzzaman Mahadi | Abrar Mohammed Tanzim Alam | Mir Mohammad Asif Abdullah | Sayma Siddika Monalisa | Swakkhar Shatabda | Md Adnan Arefeen
Proceedings of the Natural Legal Language Processing Workshop 2026
Fine-tuning improves measured accuracy on legal question answering, but does it improve how models use supplied law? We investigate this question in bilingual Bangladeshi legal QA, distinguishing gains due to scoring and retrieval from changes in reliance on the governing provision. We adapt six small instruction-tuned models with LoRA across three random seeds and evaluate them using constrained option-letter scoring, cyclic option rotation, and matched contexts that include or omit the governing provision. Scoring choices substantially change the apparent benefit of adaptation: a 50.0% gain under exact-line parsing falls to 3.0% under option scoring, and, for one model, the two methods reverse the ranking of systems. Supplied law improves accuracy, and removing the governing provision degrades performance for both base and adapted models. However, difference-in-differences estimates detect no increase in reliance on that provision after fine-tuning. These findings identify a limit to interpreting accuracy gains as evidence of legal adaptation: improved performance need not imply improved use of legal context. Evaluations must therefore distinguish scorer, retriever, and model effects. We release a hierarchy-preserving statutory corpus, 2,165 reviewed bilingual training examples, a 150-item supplied-law benchmark, and evaluation code at https://github.com/m-mahadi/bangladesh-legal-qa.
2025
NSU_PiedPiper at BLP-2025 Task 2: A Chain-of-Thought with Iterative Debugging Approach for Code Generation with Bangla Instruction
Ahmad Fahmid | Fahim Foysal | Wasif Haider | Shafin Rahman | Md Adnan Arefeen
Proceedings of the Second Workshop on Bangla Language Processing (BLP-2025)
Ahmad Fahmid | Fahim Foysal | Wasif Haider | Shafin Rahman | Md Adnan Arefeen
Proceedings of the Second Workshop on Bangla Language Processing (BLP-2025)
Code generation from natural language instructions in Bangla is a fundamental task in programming automation, as explored in BLP-2025 Shared Task 2: Code Generation in Bangla. Current code generation models are designed primarily for high-resource languages such as English, which creates major limitations when applied to Bangla. The key challenges are limited training data and difficulty in correctly interpreting Bangla programming instructions. In this paper, to accommodate Bangla instructions, we present a chain of thought (CoT) based prompting approach with Qwen2.5-Coder-14B model. We further introduce few-shot example in the prompt template to improve the accuracy. During competition, an accuracy of 93% is achieved in the shared test set (test_v1.csv) and 82.6% is achieved on the public and private test sets (hidden). After the competition is closed, we implement a debugger prompt technique which refines answers with 3 iterative fixing attempts. Applying this new technique on the public shared test set, our system outperforms by 7% and achieves 100% accuracy on the public test set, highlighting the effectiveness of combining CoT prompting with iterative debugging.