Proactive AI writing assistants need to predict when users want drafting help, yet evidence about which contextual cues drive these preferences remains limited. Through a factorial vignette study with 50 participants making 750 pairwise comparisons in professional-email scenarios, we find that compositional effort is the strongest measured predictor (𝜌 = 0.597), whereas urgency has a near-zero scenario-level correlation and no reliable effect across inferential analyses. More critically, we observe a striking stated–revealed gap between abstract factor rankings and situated choices: they rank urgency first in self-reports despite it being the weakest behavioral driver, a contrast at the extremes even though mean stated urgency and effort ranks are close, i.e., 2.06 vs. 2.18. This misalignment has measurable consequences. Within this controlled professional-email testbed, the behavior-derived rule agrees with 61.3% of the 750 observed pairwise choices, compared with 57.7% for the stated-preference rule. This 3.6-point difference corresponds to a net gain of 27 matches. It also identifies a concrete design risk for proactive email assistance: stated rankings can place weight on cues that do not predict situated choices. Future studies should test whether stated rankings create similar design risks in other writing tasks.

How can we precisely estimate a large language model’s (LLM) accuracy on questions belonging to a specific topic within a larger question-answering dataset? The standard direct estimator, which averages the model’s accuracy on the questions in each subgroup, may exhibit high variance for subgroups (topics) with small sample sizes. Synthetic regression modeling, which leverages the model’s accuracy on questions about other topics, may yield biased estimates that are too unreliable for large subgroups. We prescribe a simple yet effective solution: an empirical Bayes (EB) estimator that balances direct and regression estimates for each subgroup separately, improving the precision of subgroup-level estimates of model performance. Our experiments on multiple datasets show that this approach consistently provides more precise estimates of the LLM performance compared to the direct and regression approaches, achieving substantial reductions in the mean squared error. Confidence intervals for EB estimates also have near-nominal coverage and are narrower compared to those for the direct estimator. Additional experiments on tabular and vision data validate the benefits of this EB approach.