Andrew Konya
Author directory2026
A Contextual Measure of Ambiguity in Prescriptive Texts
Sally Dong | Luke Thorburn | Andrew Konya | Michiel A. Bakker
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Sally Dong | Luke Thorburn | Andrew Konya | Michiel A. Bakker
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Ambiguity or vagueness is a consequential consideration when drafting prescriptive text such as laws, contracts, or organizational policies. As a greater volume of such text is produced by language models, there is a need for valid, scalable methods for quantifying ambiguity, but existing measures are based on lexical properties of text rather than how it is interpreted in context. In this paper, we propose quantifying ambiguity as the degree of disagreement among a set of stakeholders ("raters") on how alternative prescriptive statements ("policies") apply in each of a set of relevant scenarios ("cases"). Using two policy domains, we present preliminary analysis suggesting that, even when using simulated raters, this approach (i) correlates with real human disagreement, and (ii) is robust to spurious text features. We then apply the measure to existing deliberation datasets to describe the association between policy ambiguity and support for those policies. A Python package for our framework and all data used in the paper are released at https://github.com/sally-dong/DELTA-ambiguity.