Shayan Bali
2025
Persian in a Court: Benchmarking VLMs In Persian Multi-Modal Tasks
Farhan Farsi
|
Shahriar Shariati Motlagh
|
Shayan Bali
|
Sadra Sabouri
|
Saeedeh Momtazi
Proceedings of the First Workshop of Evaluation of Multi-Modal Generation
This study introduces a novel framework for evaluating Large Language Models (LLMs) and Vision-Language Models (VLMs) in Persian, a low-resource language. We develop comprehensive datasets to assess reasoning, linguistic understanding, and multimodal capabilities. Our datasets include Persian-OCR-QA for optical character recognition, Persian-VQA for visual question answering, Persian world-image puzzle for multimodal integration, Visual-Abstraction-Reasoning for abstract reasoning, and Iran-places for visual knowledge of Iranian figures and locations. We evaluate models like GPT-4o, Claude 3.5 Sonnet, and Llama 3.2 90B Vision, revealing their strengths and weaknesses in processing Persian. This research contributes to inclusive language processing by addressing the unique challenges of low-resource language evaluation.
2024
RFBES at SemEval-2024 Task 8: Investigating Syntactic and Semantic Features for Distinguishing AI-Generated and Human-Written Texts
Mohammad Heydari Rad
|
Farhan Farsi
|
Shayan Bali
|
Romina Etezadi
|
Mehrnoush Shamsfard
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Nowadays, the usage of Large Language Models (LLMs) has increased, and LLMs have been used to generate texts in different languages and for different tasks. Additionally, due to the participation of remarkable companies such as Google and OpenAI, LLMs are now more accessible, and people can easily use them. However, an important issue is how we can detect AI-generated texts from human-written ones. In this article, we have investigated the problem of AI-generated text detection from two different aspects: semantics and syntax. Finally, we presented an AI model that can distinguish AI-generated texts from human-written ones with high accuracy on both multilingual and monolingual tasks using the M4 dataset. According to our results, using a semantic approach would be more helpful for detection. However, there is a lot of room for improvement in the syntactic approach, and it would be a good approach for future work.
Search
Fix data
Co-authors
- Farhan Farsi 2
- Romina Etezadi 1
- Mohammad Heydari Rad 1
- Saeedeh Momtazi 1
- Sadra Sabouri 1
- show all...