While Audio Language Models (ALMs) offer a promising shift towards explainable audio deepfake detection (ADD) via generated reasoning, whether these explanations reflect faithful perceptual grounding or fragile post-hoc rationalizations under adversarial attack remains under-explored. To address this gap, we introduce **SARA** (**S**hift **A**nalysis of **R**easoning in **A**udio), a diagnostic framework that evaluates the reasoning of five ALMs across three dimensions: acoustic perception, reasoning-verdict coherence and dissonance. Our experiments reveal three key findings. First, current ALMs exhibit weak acoustic perception (< 50% accuracy), and generating explicit rationales imposes an accuracy penalty (reducing real F1 score by up to 21%). Second, adversarial attacks show a modality asymmetry: acoustic attacks severely degrade reasoning-verdict coherence (average drop of 14.20%), whereas linguistic attacks achieve higher attack success (61.73%) while preserving fluent coherence (a drop of only 3.91%). Third, we demonstrate that decomposing reasoning shifts into prediction confidence and reasoning-verdict coherence enables a secondary defense layer that can detect adversarially perturbed audio with a 74.41% F1 score even with unseen attack strategies and speaker profiles. These findings suggest that reasoning traces also provide diagnostic utility that persists even when final classification outputs are compromised. Our code is available at [https://github.com/nqbinh17/sara_reasoning_shifts](https://github.com/nqbinh17/sara_reasoning_shifts).

Recent advances in text-to-speech technology have enabled highly realistic voice generation, fueling audio-based deepfake attacks such as fraud and impersonation. While audio anti-spoofing systems are critical for detecting such threats, prior research has predominantly focused on acoustic-level perturbations, leaving the impact of linguistic variation largely unexplored. In this paper, we investigate the linguistic sensitivity of both open-source and commercial anti-spoofing detectors by introducing TAPAS (Transcript-to-Audio Perturbation Anti-Spoofing), a novel framework for transcript-level adversarial attacks. Our extensive evaluation shows that even minor linguistic perturbations can significantly degrade detection accuracy: attack success rates exceed 60% on several open-source detector–voice pairs, and the accuracy of one commercial detector drops from 100% on synthetic audio to just 32%. Through a comprehensive feature attribution analysis, we find that linguistic complexity and model-level audio embedding similarity are key factors contributing to detector vulnerabilities. To illustrate the real-world risks, we replicate a recent Brad Pitt audio deepfake scam and demonstrate that TAPAS can bypass commercial detectors. These findings underscore the need to move beyond purely acoustic defenses and incorporate linguistic variation into the design of robust anti-spoofing systems. Our source code is available at https://github.com/nqbinh17/audio_linguistic_adversarial.