Sally Dong

Author directory

2026

Ambiguity or vagueness is a consequential consideration when drafting prescriptive text such as laws, contracts, or organizational policies. As a greater volume of such text is produced by language models, there is a need for valid, scalable methods for quantifying ambiguity, but existing measures are based on lexical properties of text rather than how it is interpreted in context. In this paper, we propose quantifying ambiguity as the degree of disagreement among a set of stakeholders ("raters") on how alternative prescriptive statements ("policies") apply in each of a set of relevant scenarios ("cases"). Using two policy domains, we present preliminary analysis suggesting that, even when using simulated raters, this approach (i) correlates with real human disagreement, and (ii) is robust to spurious text features. We then apply the measure to existing deliberation datasets to describe the association between policy ambiguity and support for those policies. A Python package for our framework and all data used in the paper are released at https://github.com/sally-dong/DELTA-ambiguity.