%0 Conference Proceedings %T DocEE: A Large-Scale and Fine-grained Benchmark for Document-level Event Extraction %A Tong, MeiHan %A Xu, Bin %A Wang, Shuai %A Han, Meihuan %A Cao, Yixin %A Zhu, Jiangqi %A Chen, Siyu %A Hou, Lei %A Li, Juanzi %Y Carpuat, Marine %Y de Marneffe, Marie-Catherine %Y Meza Ruiz, Ivan Vladimir %S Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies %D 2022 %8 July %I Association for Computational Linguistics %C Seattle, United States %F tong-etal-2022-docee %X Event extraction aims to identify an event and then extract the arguments participating in the event. Despite the great success in sentence-level event extraction, events are more naturally presented in the form of documents, with event arguments scattered in multiple sentences. However, a major barrier to promote document-level event extraction has been the lack of large-scale and practical training and evaluation datasets. In this paper, we present DocEE, a new document-level event extraction dataset including 27,000+ events, 180,000+ arguments. We highlight three features: large-scale manual annotations, fine-grained argument types and application-oriented settings. Experiments show that there is still a big gap between state-of-the-art models and human beings (41% Vs 85% in F1 score), indicating that DocEE is an open issue. DocEE is now available at https://github.com/tongmeihan1995/DocEE.git. %R 10.18653/v1/2022.naacl-main.291 %U https://aclanthology.org/2022.naacl-main.291 %U https://doi.org/10.18653/v1/2022.naacl-main.291 %P 3970-3982