Angad Ripudaman Singh Bajwa
Author directory2026
COMET Sensitivity-Guided Mixed Precision Quantization for the WMT26 Model Compression Task
Angad Ripudaman Singh Bajwa
Proceedings of the Eleventh Conference on Machine Translation
Angad Ripudaman Singh Bajwa
Proceedings of the Eleventh Conference on Machine Translation
The WMT26 Model Compression shared task asks participants to reduce the footprint of a general-purpose large language model for machine translation (MT) deployment while preserving translation quality, under a fixed evaluation budget of model size, VRAM usage, and inference speed. We report a systematic comparison of post-training compression strategies applied to the constrained track base model, google/gemma-3-12b-it, across the three required language directions (Czech–German, English–Chinese (Simplified), English–Arabic (Egyptian)). We evaluate an uncompressed BF16 baseline and a vLLM-served engine-level control against six compression strategies: two bitsandbytes baselines (8-bit and 4-bit), two calibrated weight-only quantization methods (AWQ and GPTQ, both W4A16), vLLM’s native dynamic FP8 weight quantization, and a novel COMET-sensitivity-guided mixed-precision scheme that assigns each of the 48 decoder layers an independent BF16/INT8/INT4 tier via a size-budget knapsack. Quality is measured with chrF and three COMET variants. Speed is full-process wall-clock time (including model load) on a held-out blind test set, and size is the on-disk safetensors footprint. Our mixed-precision submission achieves the best quality-size-speed trade-off in our pool: it is the fastest system measured (1.76 sentences/sec, batch 16), within 0.4 chrF and a few COMET points of the uncompressed baseline across all three language pairs, at roughly 40% of the baseline’s on-disk size (9.15GB vs 23GB).
Lost in Compression: Calibration Bias and the Accent-WER Gap in Speech LLMs
Angad Ripudaman Singh Bajwa
Proceedings of the IMPACT-SPEECH Workshop: Identifying, Measuring, Preventing, and Assessing Consequences of Bias in Speech LLMs
Angad Ripudaman Singh Bajwa
Proceedings of the IMPACT-SPEECH Workshop: Identifying, Measuring, Preventing, and Assessing Consequences of Bias in Speech LLMs
GPTQ and similar calibration-based quantization methods tune their rounding to a small calibration sample, in practice usually convenient, native-accented read speech. Edge devices disproportionately run quantized models, so the speakers most likely to depend on the compressed model are often the ones calibration represented least. Compression is known to amplify bias in text LLMs and across languages in multilingual speech models, but whether it does so within a single language, between native and non-native accents, in a modern speech LLM, remains untested. We quantize the LLM backbone of Qwen2-Audio-7B-Instruct with GPTQ, hold the recipe fixed and vary only which EdAcc speakers fill the calibration set (native-only, balanced, L2-only), measuring the native-vs-L2 word error rate (WER) gap at matched overall WER. We expected nativeonly calibration to widen the gap and balanced calibration to close it. Instead, homogeneous calibration, native-only or L2-only, narrows the gap while balanced calibration does almost nothing. It recurs when we quantize the audio encoder, weakens at 3-bit and collapses at 2-bit.