Muhammad Ali
Author directory2026
Faithfulness Is Not Free: Auditing Offline KV-Cache Quantization in Retrieval-Augmented Generation
Atta UL Asad | Ahsan Bilal | Muhammad Ali | Muhammad Haseeb | Dean F. Hougen
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Atta UL Asad | Ahsan Bilal | Muhammad Ali | Muhammad Haseeb | Dean F. Hougen
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Retrieval-augmented generation (RAG) systems can precompute and store key-value (KV) caches of retrieved documents to avoid re-encoding the same context for every query. Quantizing these offline caches reduces storage cost, but existing work largely evaluates compression through accuracy alone and does not ask whether the generated answers remain faithful to the retrieved evidence. This distinction is important because a model can preserve the correct answer string while weakening its grounding in the provided context. We audit Qwen2.5-7B-Instruct under BF16, INT8, and INT4 offline KV-cache settings on RGB and HotpotQA, measuring both answer accuracy and faithfulness using a hallucination detector, NLI entailment, and an LLM judge. INT8 is nearly lossless across accuracy and faithfulness metrics. In contrast, INT4 reduces accuracy and, more critically, degrades faithfulness even on examples where the answer remains factually correct: over 90% of faithfulness flips are negative. The degradation increases under noisier retrieval and larger retrieval depth. These results show that compressed RAG systems cannot be validated by EM or F1 alone; faithfulness must be audited before low-bit offline KV caches are deployed.
2025
Waste-Bench: A Comprehensive Benchmark for Evaluating VLLMs in Cluttered Environments
Muhammad Ali | Salman Khan
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Muhammad Ali | Salman Khan
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Recent advancements in Large Language Models (LLMs) have paved the way for VisionLarge Language Models (VLLMs) capable ofperforming a wide range of visual understand-ing tasks. While LLMs have demonstrated impressive performance on standard naturalimages, their capabilities have not been thoroughly explored in cluttered datasets where there is complex environment having deformedshaped objects. In this work, we introduce a novel dataset specifically designed for waste classification in real-world scenarios, character-ized by complex environments and deformed shaped objects. Along with this dataset, we present an in-depth evaluation approach to rig-orously assess the robustness and accuracy of VLLMs. The introduced dataset and comprehensive analysis provide valuable insights intothe performance of VLLMs under challenging conditions. Our findings highlight the critical need for further advancements in VLLM’s ro-bustness to perform better in complex enviroments. The dataset and code for our experiments are available at https://github.com/aliman80/wastebench.
2024
Antonym vs Synonym Distinction using InterlaCed Encoder NETworks (ICE-NET)
Muhammad Ali | Yan Hu | Jianbin Qin | Di Wang
Findings of the Association for Computational Linguistics: EACL 2024
Muhammad Ali | Yan Hu | Jianbin Qin | Di Wang
Findings of the Association for Computational Linguistics: EACL 2024
Antonyms vs synonyms distinction is a core challenge in lexico-semantic analysis and automated lexical resource construction. These pairs share a similar distributional context which makes it harder to distinguish them. Leading research in this regard attempts to capture the properties of the relation pairs, i.e., symmetry, transitivity, and trans-transitivity. However, the inability of existing research to appropriately model the relation-specific properties limits their end performance. In this paper, we propose InterlaCed Encoder NETworks (i.e., ICE-NET) for antonym vs synonym distinction, that aim to capture and model the relation-specific properties of the antonyms and synonyms pairs in order to perform the classification task in a performance-enhanced manner. Experimental evaluation using the benchmark datasets shows that ICE-NET outperforms the existing research by a relative score of upto 1.8% in F1-measure.
2023
GRI: Graph-based Relative Isomorphism of Word Embedding Spaces
Muhammad Ali | Yan Hu | Jianbin Qin | Di Wang
Findings of the Association for Computational Linguistics: EMNLP 2023
Muhammad Ali | Yan Hu | Jianbin Qin | Di Wang
Findings of the Association for Computational Linguistics: EMNLP 2023
Automated construction of bi-lingual dictionaries using monolingual embedding spaces is a core challenge in machine translation. The end performance of these dictionaries relies on the geometric similarity of individual spaces, i.e., their degree of isomorphism. Existing attempts aimed at controlling the relative isomorphism of different spaces fail to incorporate the impact of lexically different but semantically related words in the training objective. To address this, we propose GRI that combines the distributional training objectives with attentive graph convolutions to unanimously consider the impact of lexical variations of semantically similar words required to define/compute the relative isomorphism of multiple spaces. Exper imental evaluation shows that GRI outperforms the existing research by improving the average P@1 by a relative score of upto 63.6%.
GARI: Graph Attention for Relative Isomorphism of Arabic Word Embeddings
Muhammad Ali | Maha Alshmrani | Jianbin Qin | Yan Hu | Di Wang
Proceedings of ArabicNLP 2023
Muhammad Ali | Maha Alshmrani | Jianbin Qin | Yan Hu | Di Wang
Proceedings of ArabicNLP 2023
Bilingual Lexical Induction (BLI) is a core challenge in NLP, it relies on the relative isomorphism of individual embedding spaces. Existing attempts aimed at controlling the relative isomorphism of different embedding spaces fail to incorporate the impact of semantically related words in the model training objective. To address this, we propose GARI that combines the distributional training objectives with multiple isomorphism losses guided by the graph attention network. GARI considers the impact of semantical variations of words in order to define the relative isomorphism of the embedding spaces. Experimental evaluation using the Arabic language data set shows that GARI outperforms the existing research by improving the average P@1 by a relative score of up to 40.95% and 76.80% for in-domain and domain mismatch settings respectively.