New Insights into Cross-Document Event Coreference: Systematic Comparison and a Simplified Approach

Andres Cremisini, Mark Finlayson


Abstract
Cross-Document Event Coreference (CDEC) is the task of finding coreference relationships between events in separate documents, most commonly assessed using the Event Coreference Bank+ corpus (ECB+). At least two different approaches have been proposed for CDEC on ECB+ that use only event triggers, and at least four have been proposed that use both triggers and entities. Comparing these approaches is complicated by variation in the systems’ use of gold vs. computed labels, as well as variation in the document clustering pre-processing step. We present an approach that matches or slightly beats state-of-the-art performance on CDEC over ECB+ with only event trigger annotations, but with a significantly simpler framework and much smaller feature set relative to prior work. This study allows us to directly compare with prior systems and draw conclusions about the effectiveness of various strategies. Additionally, we provide the first cross-validated evaluation on the ECB+ dataset; the first explicit evaluation of the pairwise event coreference classification step; and the first quantification of the effect of document clustering on system performance. The last in particular reveals that while document clustering is a crucial pre-processing step, improvements can at most provide for a 3 point improvement in CDEC performance, though this might be attributable to ease of document clustering on ECB+.
Anthology ID:
2020.nuse-1.1
Volume:
Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events
Month:
July
Year:
2020
Address:
Online
Editors:
Claire Bonial, Tommaso Caselli, Snigdha Chaturvedi, Elizabeth Clark, Ruihong Huang, Mohit Iyyer, Alejandro Jaimes, Heng Ji, Lara J. Martin, Ben Miller, Teruko Mitamura, Nanyun Peng, Joel Tetreault
Venues:
NUSE | WNU
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–10
Language:
URL:
https://aclanthology.org/2020.nuse-1.1
DOI:
10.18653/v1/2020.nuse-1.1
Bibkey:
Cite (ACL):
Andres Cremisini and Mark Finlayson. 2020. New Insights into Cross-Document Event Coreference: Systematic Comparison and a Simplified Approach. In Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events, pages 1–10, Online. Association for Computational Linguistics.
Cite (Informal):
New Insights into Cross-Document Event Coreference: Systematic Comparison and a Simplified Approach (Cremisini & Finlayson, NUSE-WNU 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.nuse-1.1.pdf
Video:
 http://slideslive.com/38929739