Joe Ellis


2018

pdf bib
Laying the Groundwork for Knowledge Base Population: Nine Years of Linguistic Resources for TAC KBP
Jeremy Getman | Joe Ellis | Stephanie Strassel | Zhiyi Song | Jennifer Tracey
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)

2016

pdf bib
A Comparison of Event Representations in DEFT
Ann Bies | Zhiyi Song | Jeremy Getman | Joe Ellis | Justin Mott | Stephanie Strassel | Martha Palmer | Teruko Mitamura | Marjorie Freedman | Heng Ji | Tim O’Gorman
Proceedings of the Fourth Workshop on Events

pdf bib
Event Nugget and Event Coreference Annotation
Zhiyi Song | Ann Bies | Stephanie Strassel | Joe Ellis | Teruko Mitamura | Hoa Trang Dang | Yukari Yamakawa | Sue Holm
Proceedings of the Fourth Workshop on Events

2015

pdf bib
From Light to Rich ERE: Annotation of Entities, Relations, and Events
Zhiyi Song | Ann Bies | Stephanie Strassel | Tom Riese | Justin Mott | Joe Ellis | Jonathan Wright | Seth Kulick | Neville Ryant | Xiaoyi Ma
Proceedings of the 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation

2014

pdf bib
A Comparison of the Events and Relations Across ACE, ERE, TAC-KBP, and FrameNet Annotation Standards
Jacqueline Aguilar | Charley Beller | Paul McNamee | Benjamin Van Durme | Stephanie Strassel | Zhiyi Song | Joe Ellis
Proceedings of the Second Workshop on EVENTS: Definition, Detection, Coreference, and Representation

2012

pdf bib
Linguistic Resources for Entity Linking Evaluation: from Monolingual to Cross-lingual
Xuansong Li | Stephanie Strassel | Heng Ji | Kira Griffitt | Joe Ellis
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)

To advance information extraction and question answering technologies toward a more realistic path, the U.S. NIST (National Institute of Standards and Technology) initiated the KBP (Knowledge Base Population) task as one of the TAC (Text Analysis Conference) evaluation tracks. It aims to encourage research in automatic information extraction of named entities from unstructured texts with the ultimate goal of integrating such information into a structured Knowledge Base. The KBP track consists of two types of evaluation: Named Entity Linking (NEL) and Slot Filling. This paper describes the linguistic resource creation efforts at the Linguistic Data Consortium (LDC) in support of Named Entity Linking evaluation of KBP, focusing on annotation methodologies, process, and features of corpora from 2009 to 2011, with a highlighted analysis of the cross-lingual NEL data. Progressing from monolingual to cross-lingual Entity Linking technologies, the 2011 cross-lingual NEL evaluation targeted multilingual capabilities. Annotation accuracy is presented in comparison with system performance, with promising results from cross-lingual entity linking systems.

pdf bib
Annotation Trees: LDC’s customizable, extensible, scalable, annotation infrastructure
Jonathan Wright | Kira Griffitt | Joe Ellis | Stephanie Strassel | Brendan Callahan
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)

In recent months, LDC has developed a web-based annotation infrastructure centered around a tree model of annotations and a Ruby on Rails application called the LDC User Interface (LUI). The effort aims to centralize all annotation into this single platform, which means annotation is always available remotely, with no more software required than a web browser. While the design is monolithic in the sense of handling any number of annotation projects, it is also scalable, as it is distributed over many physical and virtual machines. Furthermore, minimizing customization was a core design principle, and new functionality can be plugged in without writing a full application. The creation and customization of GUIs is itself done through the web interface, without writing code, with the aim of eventually allowing project managers to create a new task without developer intervention. Many of the desirable features follow from the model of annotations as trees, and the operationalization of annotation as tree modification.