Building Evaluation Datasets for Consumer-Oriented Information Retrieval

Lorraine Goeuriot, Liadh Kelly, Guido Zuccon, Joao Palotti


Abstract
Common people often experience difficulties in accessing relevant, correct, accurate and understandable health information online. Developing search techniques that aid these information needs is challenging. In this paper we present the datasets created by CLEF eHealth Lab from 2013-2015 for evaluation of search solutions to support common people finding health information online. Specifically, the CLEF eHealth information retrieval (IR) task of this Lab has provided the research community with benchmarks for evaluating consumer-centered health information retrieval, thus fostering research and development aimed to address this challenging problem. Given consumer queries, the goal of the task is to retrieve relevant documents from the provided collection of web pages. The shared datasets provide a large health web crawl, queries representing people’s real world information needs, and relevance assessment judgements for the queries.
Anthology ID:
L16-1306
Volume:
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
Month:
May
Year:
2016
Address:
Portorož, Slovenia
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
1932–1938
Language:
URL:
https://aclanthology.org/L16-1306
DOI:
Bibkey:
Cite (ACL):
Lorraine Goeuriot, Liadh Kelly, Guido Zuccon, and Joao Palotti. 2016. Building Evaluation Datasets for Consumer-Oriented Information Retrieval. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 1932–1938, Portorož, Slovenia. European Language Resources Association (ELRA).
Cite (Informal):
Building Evaluation Datasets for Consumer-Oriented Information Retrieval (Goeuriot et al., LREC 2016)
Copy Citation:
PDF:
https://aclanthology.org/L16-1306.pdf