On maximizing metrics for syntactic disambiguation

Khalil Sima’an


Abstract
Given a probabilistic parsing model and an evaluation metric for scoring the match between parse-trees, e.g., PARSEVAL [Black et al., 1991], this paper addresses the problem of how to select the on average best scoring parse-tree for an input sentence. Common wisdom dictates that it is optimal to select the parse with the highest probability, regardless of the evaluation metric. In contrast, the Maximizing Metrics (MM) method [Goodman, 1998, Stolcke et al., 1997] proposes that an algorithm that optimizes the evaluation metric itself constitutes the optimal choice. We study the MM method within parsing. We observe that the MM does not always hold for tree-bank models, and that optimizing weak metrics is not interesting for semantic processing. Subsequently, we state an alternative proposition: the optimal algorithm must maximize the metric that scores parse-trees according to linguistically relevant features. We present new algorithms that optimize metrics that take into account increasingly more linguistic features, and exhibit experiments in support of our claim.
Anthology ID:
W03-3021
Volume:
Proceedings of the Eighth International Conference on Parsing Technologies
Month:
April
Year:
2003
Address:
Nancy, France
Venue:
IWPT
SIG:
SIGPARSE
Publisher:
Note:
Pages:
183–194
Language:
URL:
https://aclanthology.org/W03-3021
DOI:
Bibkey:
Cite (ACL):
Khalil Sima’an. 2003. On maximizing metrics for syntactic disambiguation. In Proceedings of the Eighth International Conference on Parsing Technologies, pages 183–194, Nancy, France.
Cite (Informal):
On maximizing metrics for syntactic disambiguation (Sima’an, IWPT 2003)
Copy Citation:
PDF:
https://aclanthology.org/W03-3021.pdf