Suggestion mining has traditionally been formulated as binary classification, but the underlying linguistic phenomenon is not binary. A single review can express the same operational recommendation through a direct imperative (e.g., “add a vegetarian entrée”), an optative wish, an interrogative request, or a hedged conditional. These utterances share illocutionary force but differ substantially in pragmatic form, placing different demands on models: lexical cues may suffice for direct suggestions, whereas indirect forms require pragmatic interpretation. Existing benchmarks, typically spanning one to five domains and using a single binary label, do not measure these distinctions.

We introduce PRAXIS, an English consumer-review benchmark that reframes suggestion mining as the measurement of pragmatic-form bias rather than as a leaderboard task. PRAXIS provides (i) a silver corpus of 174,860 reviews spanning 68 industries, labeled by Claude Opus 4.7; (ii) a 15,000-example human-labeled gold evaluation set, with annotation reliability estimated by a three-annotator panel on a 1,000-example subset (Tier 1 Fleiss’ κ = 0.77); and (iii) evaluations of four model families against the human gold, with bootstrap 95% confidence intervals. Agreement between Opus and human labels is reported separately as a corpus-validation statistic (Tier 1 Cohen’s κ = 0.71), rather than as inter-annotator agreement.

Our central finding is structural rather than competitive. Instruction-tuned LLMs outperform encoders on binary macro-F1 (DeepSeek-V4-Pro: 0.882 vs. RoBERTa-large: 0.780), yet the ranking reverses for the Tier 1 conditional form: RoBERTa-large recalls 85.2% of conditional suggestions, compared with 46.7% for the strongest zero-shot LLM and 8.0–43.1% for the other six LLMs. This discrepancy reflects a form-classification gap rather than a detection gap: models often recognize these instances as suggestions at the binary level but misclassify their pragmatic form. A matched-supervision QLoRA model based on Qwen2.5-7B, trained on the same silver labels used for the encoders, recovers encoder-level conditional recall (0.904). PRAXIS therefore exposes a structural error mode that flat binary metrics obscure, positioning the benchmark as a measurement instrument for pragmatic-form bias rather than a conventional model leaderboard.