Annotating Inter-Sentence Temporal Relations in Clinical Notes

Jennifer D’Souza, Vincent Ng


Abstract
Owing in part to the surge of interest in temporal relation extraction, a number of datasets manually annotated with temporal relations between event-event pairs and event-time pairs have been produced recently. However, it is not uncommon to find missing annotations in these manually annotated datasets. Many researchers attributed this problem to “annotator fatigue”. While some of these missing relations can be recovered automatically, many of them cannot. Our goals in this paper are to (1) manually annotate certain types of missing links that cannot be automatically recovered in the i2b2 Clinical Temporal Relations Challenge Corpus, one of the recently released evaluation corpora for temporal relation extraction; and (2) empirically determine the usefulness of these additional annotations. We will make our annotations publicly available, in hopes of enabling a more accurate evaluation of temporal relation extraction systems.
Anthology ID:
L14-1129
Volume:
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Month:
May
Year:
2014
Address:
Reykjavik, Iceland
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
2758–2765
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/1185_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Jennifer D’Souza and Vincent Ng. 2014. Annotating Inter-Sentence Temporal Relations in Clinical Notes. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 2758–2765, Reykjavik, Iceland. European Language Resources Association (ELRA).
Cite (Informal):
Annotating Inter-Sentence Temporal Relations in Clinical Notes (D’Souza & Ng, LREC 2014)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/1185_Paper.pdf