MT EVALUATION: BASIS FOR FUTURE
DIRECTIONS
AN NSF-SPONSORED WORKSHOP
(San Diego, 2-3
November 1992)
PROGRAM
and Proceedings
Preface—M.Vasconcellos
Monday, 2 November
Announcements and Greetings
Association
for Machine Translation in the
European
Association for Machine Translation—M. King
National
Science Foundation—Y.T. Chien
Traditions in the Evaluation of MT—Y.
Wilks
International Coordination
International Working Group on MT
Evaluation—M. King
EC Evaluation Activities—L.
Rolling
Evaluation Surveys
The JEIDA Methodology and Survey—H. Nomura,
H. Isahara
The MITRE Methodology and Survey—P. Jordan, J. Benoit, B. Dorr
The DARPA Methodology
Objective—G.Doddington
Presentation—J. White
Participants—P. Brown, L.
Carlson, G. Doddington, D. Farwell, E. Hovy, E. Lange,
M. Leon, M. Mandel, S. Nirenburg
Outside commentators—R. Berwick,
L. Rolling, M. Vasconcellos
New Directions
It’s Lucky If Concatenation Does Anything—R.
Berwick
Automatic Evaluation of Translation Quality—H. Thompson
Evaluation of
the Machine-Aided
Voice Translation System—
Evaluation
of Related NLP Research
Message Understanding: The MUC Methodology—B. Sundheim
Semi-Automated Evaluation of Japanese and English Output—L. Carlson
The Murasaki Project: Multilingual Natural Language Understanding—R. Doerr
Measurement of Human Performance for Information Extraction—C. Will, B. Onyshkevych
Tuesday, 3 November
Discussion:
"Apples,
M. Vasconcellos (moderator), R. Dessau, E. Hovy, V. Lawson, C. Miller, M.
Nagao, B. Scott, M.Tacelosky
Practical Use and Effect of the JICST Japanese-English Machine
Translation System—K. Tamura,
LOGOS— B. Scott
Methodology for the Computing Research Laboratory’s Multilingual MT
System—D. Farwell
Quality Analysis, an Internal Evaluation Tool at SYSTRAN—E. Lange, J. Homer,
L. Gerber
METAL—W. S. Bennett
Approaches to the Evaluation of
Japanese-English MT
Criteria for Evaluating the Linguistic Quality of
Japanese-English MT—S. Ikehara
Application of the TOEFL Test to the Evaluation of
Japanese-English MT—M. Tomita [abstract]
Evaluation of Lexical Coverage— G.
Magnusdottir
Discussion:
Assessing the Labor-intensive Elements of MT—M. León (moderator), D. Albisser, M. Baena, L.
Childs, G. Magnusdottir
Results of the Warmup Exercises—M. León
Discussion:
Future Directions in MT—S.
Nirenburg (moderator), W.S. Bennett, B. Dorr, E. Hovy, S. Ikehara, M. King, A. Melby, M. Nagao, M. Tomita, V. Teller
Closing Messages
IAMT—M. Nagao
AMTA—M.
Vasconcellos
Annex A:
Machine
Translation Evaluation Criteria—R.Dessau
Annex B:
Warmup exercises [introduction]
Photocopy of: Machine Translation
Evaluation Methodology [
Photocopy of: Appendix 10 of the ALPAC report: An experiment in evaluating the quality of
translations