Rishabh Kartik Jawagal

Author directory

2026

Automated legal citation verification systems are critical for ensuring the factual integrity of AI-assisted legal research and drafting, but large language models (LLMs) serving as verifiers remain vulnerable to adversarial persuasion. We introduce a three-class benchmark dataset (valid, real_wrong_content, fabricated) derived from CourtListener opinions via a hybrid extraction and dual-model validation pipeline, and evaluate two model families (Llama, GPT-OSS) plus a cross-family adversarial pairing across control, authority-appeal, and fabricated-citation conditions, both ungrounded and retrieval-grounded. Within-model logistic regressions with item-clustered robust standard errors show that authority appeals massively increase the odds of a correct GPT-OSS 20B verification flipping to incorrect (OR=23.333, p<0.0001) and show strong effects in Llama 70B (OR=11.953, p=0.0029), avoiding small-model contamination. Retrieval grounding broadly reduces flip rates in large models (GPT-OSS 120B OR_Grounded=0.123; Llama 70B OR_Grounded=0.239). Cross-family attacks (GPT-OSS 120B → Llama 70B) achieve 98.0% ungrounded authority true_ASR vs. a 70.6% control baseline, and grounding does not eliminate this gap (+39.7pp, p<0.000001). Beyond these findings, we argue that persuasion benchmarks require a content-free control condition and a Confidence-Weighted Attack Success Rate (CW-ASR) to separate genuine "confident override" from "hesitant doubt": authority appeals make verifiers confidently wrong (mean final confidence 8.56/10; 96.8% of flips at confidence ≥ 8), whereas content-free reconsideration produces hesitant, low-confidence reversals (mean 6.80/10).