%0 Conference Proceedings %T HEAD-QA: A Healthcare Dataset for Complex Reasoning %A Vilares, David %A Gómez-Rodríguez, Carlos %Y Korhonen, Anna %Y Traum, David %Y Màrquez, Lluís %S Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics %D 2019 %8 July %I Association for Computational Linguistics %C Florence, Italy %F vilares-gomez-rodriguez-2019-head %X We present HEAD-QA, a multi-choice question answering testbed to encourage research on complex reasoning. The questions come from exams to access a specialized position in the Spanish healthcare system, and are challenging even for highly specialized humans. We then consider monolingual (Spanish) and cross-lingual (to English) experiments with information retrieval and neural techniques. We show that: (i) HEAD-QA challenges current methods, and (ii) the results lag well behind human performance, demonstrating its usefulness as a benchmark for future work. %R 10.18653/v1/P19-1092 %U https://aclanthology.org/P19-1092 %U https://doi.org/10.18653/v1/P19-1092 %P 960-966