Measure for Measure: Parser Cross-fertilization - Towards Increased Component Comparability and Exchange

Stephan Oepen, Ulrich Callmeier


Abstract
Over the past few years significant progress was accomplished in efficient processing with wide-coverage HPSG grammars. HPSG-based parsing systems are now available that can process medium-complexity sentences (of ten to twenty words, say) in average parse times equivalent to real (i.e. human reading) time. A large number of engineering improvements in current HPSG systems were achieved through collaboration of multiple research centers and mutual exchange of experience, encoding techniques, algorithms, and even pieces of software. This article presents an approach to grammar and system engineering, termed competence & performance profiling, that makes systematic experimentation and the precise empirical study of system properties a focal point in development. Adapting the profiling metaphor familiar from software engineering to constraint-based grammars and parsers, enables developers to maintain an accurate record of system evolution, identify grammar and system deficiencies quickly, and compare to earlier versions or between different systems. We discuss a number of exemplary problems that motivate the experimental approach, and apply the empirical methodology in a fairly detailed discussion of what was achieved during a development period of three years. Given the collaborative nature in setup, the empirical results we present involve research and achievements of a large group of people.
Anthology ID:
2000.iwpt-1.19
Volume:
Proceedings of the Sixth International Workshop on Parsing Technologies
Month:
February 23-25
Year:
2000
Address:
Trento, Italy
Editors:
Alberto Lavelli, John Carroll, Robert C. Berwick, Harry C. Bunt, Bob Carpenter, John Carroll, Ken Church, Mark Johnson, Aravind Joshi, Ronald Kaplan, Martin Kay, Bernard Lang, Alon Lavie, Anton Nijholt, Christer Samuelsson, Mark Steedman, Oliviero Stock, Hozumi Tanaka, Masaru Tomita, Hans Uszkoreit, K. Vijay-Shanker, David Weir, Mats Wiren
Venue:
IWPT
SIG:
SIGPARSE
Publisher:
Association for Computational Linguistics
Note:
Pages:
183–194
Language:
URL:
https://aclanthology.org/2000.iwpt-1.19
DOI:
Bibkey:
Cite (ACL):
Stephan Oepen and Ulrich Callmeier. 2000. Measure for Measure: Parser Cross-fertilization - Towards Increased Component Comparability and Exchange. In Proceedings of the Sixth International Workshop on Parsing Technologies, pages 183–194, Trento, Italy. Association for Computational Linguistics.
Cite (Informal):
Measure for Measure: Parser Cross-fertilization - Towards Increased Component Comparability and Exchange (Oepen & Callmeier, IWPT 2000)
Copy Citation:
PDF:
https://aclanthology.org/2000.iwpt-1.19.pdf