Frank Lin


pdf bib
Modular Approach to Error Analysis and Evaluation for Multilingual Question Answering
Hideki Shima | Mengqiu Wang | Frank Lin | Teruko Mitamura
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)

Multilingual Question Answering systems are generally very complex, integrating several sub-modules to achieve their result. Global metrics (such as average precision and recall) are insufficient when evaluating the performance of individual sub-modules and their influence on each other. In this paper, we present a modular approach to error analysis and evaluation; we use manually-constructed, gold-standard input for each module to obtain an upper-bound for the (local) performance of that module. This approach enables us to identify existing problem areas quickly, and to target improvements accordingly.

pdf bib
Keyword Translation Accuracy and Cross-Lingual Question Answering inChinese and Japanese
Teruko Mitamura | Mengqiu Wang | Hideki Shima | Frank Lin
Proceedings of the Workshop on Multilingual Question Answering - MLQA ‘06


pdf bib
Keyword translation from English to Chinese for multilingual QA
Frank Lin | Teruko Mitamura
Proceedings of the 6th Conference of the Association for Machine Translation in the Americas: Technical Papers

The Keyword Translator is a part of the Question Analyzer module in the JAVELIN Question-Answering system; it translates the keywords, which are used to query documents and extract answers, from one language to another. Much work has been in the area of query translation for CLIR or MLIR, however, many have focused on methods using hard-to-obtain and domain-specific resources, and evaluation is often based on retrieval performance rather than translation correctness. In this paper we will describe methods combining easily accessible, general-purpose MT systems to improve keyword translation correctness. We also describe methods that utilize the question sentence available to a question-answering system to improve translation correctness. We will show that using multiple MT systems and the question sentence to translate keywords from English to Mandarin Chinese can significantly improve keyword translation correctness.