Overview of Todai Robot Project and Evaluation Framework of its NLP-based Problem Solving

Akira Fujita, Akihiro Kameda, Ai Kawazoe, Yusuke Miyao


Abstract
We introduce the organization of the Todai Robot Project and discuss its achievements. The Todai Robot Project task focuses on benchmarking NLP systems for problem solving. This task encourages NLP-based systems to solve real high-school examinations. We describe the details of the method to manage question resources and their correct answers, answering tools and participation by researchers in the task. We also analyse the answering accuracy of the developed systems by comparing the systems’ answers with answers given by human test-takers.
Anthology ID:
L14-1580
Volume:
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Month:
May
Year:
2014
Address:
Reykjavik, Iceland
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
2590–2597
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/74_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Akira Fujita, Akihiro Kameda, Ai Kawazoe, and Yusuke Miyao. 2014. Overview of Todai Robot Project and Evaluation Framework of its NLP-based Problem Solving. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 2590–2597, Reykjavik, Iceland. European Language Resources Association (ELRA).
Cite (Informal):
Overview of Todai Robot Project and Evaluation Framework of its NLP-based Problem Solving (Fujita et al., LREC 2014)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/74_Paper.pdf