Suggestion mining has traditionally been formulated as binary classification, but the underlying linguistic phenomenon is not binary. A single review can express the same operational recommendation through a direct imperative (e.g., “add a vegetarian entrée”), an optative wish, an interrogative request, or a hedged conditional. These utterances share illocutionary force but differ substantially in pragmatic form, placing different demands on models: lexical cues may suffice for direct suggestions, whereas indirect forms require pragmatic interpretation. Existing benchmarks, typically spanning one to five domains and using a single binary label, do not measure these distinctions.

We introduce PRAXIS, an English consumer-review benchmark that reframes suggestion mining as the measurement of pragmatic-form bias rather than as a leaderboard task. PRAXIS provides (i) a silver corpus of 174,860 reviews spanning 68 industries, labeled by Claude Opus 4.7; (ii) a 15,000-example human-labeled gold evaluation set, with annotation reliability estimated by a three-annotator panel on a 1,000-example subset (Tier 1 Fleiss’ κ = 0.77); and (iii) evaluations of four model families against the human gold, with bootstrap 95% confidence intervals. Agreement between Opus and human labels is reported separately as a corpus-validation statistic (Tier 1 Cohen’s κ = 0.71), rather than as inter-annotator agreement.

Our central finding is structural rather than competitive. Instruction-tuned LLMs outperform encoders on binary macro-F1 (DeepSeek-V4-Pro: 0.882 vs. RoBERTa-large: 0.780), yet the ranking reverses for the Tier 1 conditional form: RoBERTa-large recalls 85.2% of conditional suggestions, compared with 46.7% for the strongest zero-shot LLM and 8.0–43.1% for the other six LLMs. This discrepancy reflects a form-classification gap rather than a detection gap: models often recognize these instances as suggestions at the binary level but misclassify their pragmatic form. A matched-supervision QLoRA model based on Qwen2.5-7B, trained on the same silver labels used for the encoders, recovers encoder-level conditional recall (0.904). PRAXIS therefore exposes a structural error mode that flat binary metrics obscure, positioning the benchmark as a measurement instrument for pragmatic-form bias rather than a conventional model leaderboard.

Detecting suggestions in reviews forces a trade- off: an LLM on every review is slow and costly; a cheap classifier is fast but its error rate is uncontrolled, measurable after the fact, not settable in advance. We present a certified triage layer: a calibration step computes ex- actly where a classifier may act alone so its error there stays below an operator-chosen tar- get α; the rest is escalated. On a 14,995-review gold set over 68 industry categories, natural- weighted to a ∼9.5% deployment positive rate, it auto-handles 74–93% of the stream at cer- tified error for α ∈ 2, 4, 5, 10%. Heuris- tic thresholding violates the α=5% target on 36–49% of draws and an in-expectation con- formal gate on 25–28%; only a betting certifi- cate bounds the violation rate by δ, making a strict 2% SLA rigorous at 74% order-free cover- age. Cost-sensitive training, not threshold tun- ing, lowers the certifiable error floor to 0.39%, and ∼200 target labels restore it under cross- industry shift. Of 13 open-weight LLMs (20B– 675B), 11 are worse detectors than our fine- tuned 435M encoder, supervision, not scale, is why (within-model control, §5.6), and the cer- tificate rides on either backbone. Escalation targets a human: on the deferred slice the best LLM oracle still errs at 21.5%.

Extracting actionable suggestions from customer reviews is essential for operational decision-making, yet these directives are often embedded within mixed-intent, unstructured text. Existing approaches either classify suggestion-bearing sentences or generate high-level summaries, but rarely isolate the precise improvement instructions businesses need.We evaluate a hybrid pipeline combining a high-recall RoBERTa classifier trained with a precision–recall surrogate to reduce unrecoverable false negatives with a controlled, instruction-tuned LLM for suggestion extraction, categorization, clustering, and summarization. Across real-world hospitality and food datasets, the hybrid system outperforms prompt-only, rule-based, and classifier-only baselines in extraction accuracy and cluster coherence. Human evaluations further confirm that the resulting suggestions and summaries are clear, faithful, and interpretable.Overall, our results show that hybrid reasoning architectures achieve meaningful improvements fine-grained actionable suggestion mining while highlighting challenges in domain adaptation and efficient local deployment.