Event-Centric Natural Language Processing

This tutorial targets researchers and practitioners who are interested in AI technologies that help machines understand natural language text, particularly real-world events described in the text. These include methods to extract the internal structures of an event regarding its protagonist(s), participant(s) and properties, as well as external structures concerning memberships, temporal and causal relations of multiple events. This tutorial will provide audience with a systematic introduction of (i) knowledge representations of events, (ii) various methods for automated extraction, conceptualization and prediction of events and their relations, (iii) induction of event processes and properties, and (iv) a wide range of NLU and commonsense understanding tasks that benefit from aforementioned techniques. We will conclude the tutorial by outlining emerging research problems in this area.


Introduction
Human languages always involve the description of real-world events. Therefore, understanding events plays a critical role in NLP. For example, narrative prediction benefits from learning the causal relations of events to predict what happens next in a story (Chaturvedi et al., 2017a); machine comprehension of documents may involve understanding of events that affect the stock market (Ding et al., 2015), describe natural phenomena (Berant et al., 2014) or identify disease phenotypes (Zhang et al., 2020d). In fact, event understanding also widely finds its important use cases in tasks such as opendomain question answering (Yang et al., 2003), intent prediction (Rashkin et al., 2018), timeline construction (Do et al., 2012), text summarization (Daumé III and Marcu, 2006) and misinformation detection (Fung et al., 2021). Since events are not just simple, standalone predicates, frontier research on event understanding generally faces two key challenges. One challenge is to precisely induce the relations of events, which describe memberships, co-reference, temporal orders and causality of events. The other is to comprehend the inherent structure and properties of an event, concerning its participants, granularity, location and time.
In this tutorial, we will comprehensively review existing paradigms for event-centric knowledge representation in literature, and focus on their contributions to NLP tasks. Beyond introducing partial-label and unsupervised learning approaches for event extraction, we will discuss recent constrained learning and structured inference approaches for multi-faceted event-event relation extraction from text. We will also review recent data-driven methods for event prediction tasks, including event process induction and conceptualization, and how an event-centric language model benefits narrative prediction. In addition, we will illustrate how distantly supervised approaches help resolve temporal and causal commonsense understanding of events, and how they can be applied to construct a large-scale eventuality knowledge base. Participants will learn about recent trends and emerging challenges in this topic, representative tools and learning resources to obtain ready-to-use models, and how related models and techniques benefit end-use NLP applications.

Outline of Tutorial Content
This half-day tutorial presents a systematic overview of recent advances in event-centric NLP technologies. We will begin with motivating this topic with several real-world applications, and introduce the main research problems. Then, we will introduce methods for automated extraction of events as well as their participants, properties and relations from natural language text. Based on the extracted eventuality knowledge, we will explain how various prediction tasks, including the completion of an event complex, conceptualization and consolidation of event processes, can be resolved. We will also discuss commonsense understanding of events, with a focus on the temporal and cognitive aspects. Moreover, we will exemplify the use of aforementioned technologies in NLP applications of various domains, and will outline emerging research challenges that may catalyze further investigation on this topic. The detailed contents are outlined below.

Motivation [20min]
We will define the main research problem and motivate the topic by presenting several real-world applications based on event-centric NLP. This seeks to provide 20 minutes of presented content to motivate the main topic of this tutorial.

Background of Events and Their Representations [30min]
We will start the tutorial by introducing the essential background knowledge about events and their relations, including the definitions, categorizations, and applications (P. D. Mourelatos, 1978;Bach, 1986). In the last part of the introduction, we will talk about widely used event representation methods, including event schemas (Baker et al., 1998;Li et al., 2020bLi et al., , 2021a, event knowledge graphs (Zhang et al., 2020c), event processes (Chambers and Jurafsky, 2008), event language models (Peng et al., 2017), and more recent work on event meaning representation via questionanswer pairs (He et al., 2015;Michael et al., 2018), event network embeddings (Zeng et al., 2021) and event time expression embeddings (Goyal and Durrett, 2019). This part is estimated to take 30 minutes.

Event-centric Information Extraction [40min]
We will introduce unsupervised and zero-shot techniques for parsing the internal structures of verb and nominal events from natural language text, which also involves methods for automatic salient event detection (Liu et al., 2018), joint entity, relation and event extraction (Lin et al., 2020), and graph neural networks based encoding and decoding for information extraction (Zhang and Ji, 2021). Then we will discuss the recent research trend to extend information extraction from sentence-level to document-level (Du and Cardie, 2020;Li et al., 2021b). Besides, we will also discuss methods that identify temporal and causal relations of primitive events (Ning et al., 2018), and membership relations of multi-granular events (Aldawsari and Finlayson, 2019). Specifically, for data-driven extraction methods, we will present how constrained learning (Li et al., 2019) and structured prediction are incorporated to improve the tasks by enforcing logic consistency among different categories of event-event relations (Wang et al., 2020). We will also cover various cross-domain (Huang et al., 2018), cross-lingual (Subburathinam et al., 2019) and cross-media (Li et al., 2020a) structure transfer approaches for event extraction. This part is estimated to be 40 minutes.

Understanding Event Processes [35min]
We will then present recent works on machine comprehension and prediction on event processes/sequences. Specifically, people are trying to understand the progress of events from different angles. For example, many efforts have been devoted into modeling event narratives (Peng et al., 2017;Chaturvedi et al., 2017b;Lee and Goldwasser, 2019) such that they can successfully predict missing events in an event process. Besides, another important event understanding angle is conceptualization (Zhang et al., 2020a), which aims at understanding the super-sub relations between a coarse-grained event and a fine-grained event process (Glavaš et al., 2014). In this context, the machine could also be expected to generate the event process given a goal (Zhang et al., 2020a), infer the goal given the process (Chen et al., 2020), and capture the recurrence of events in a process (Zhu et al., 2021). Last but not least, event coreference, which links references to the same event together, also plays a critical role in understanding events (Cybulska and Vossen, 2014). This part should last for 35 minutes.

Event-centric Commonsense Knowledge Acquisition [35min]
Commonsense reasoning is a challenging yet important research problem in the AI community and one key challenge we are facing is the lack of satisfactory commonsense knowledge resources about events. Previous resources (Liu and Singh, 2004) typically require laborious and expensive human annotations, which are not feasible on a large scale. In this tutorial, we introduce recent progress on modeling commonsense knowledge with high-order selectional preference over event knowledge and demonstrates that how to convert relatively cheap event knowledge, which can be easily acquired from raw documents with linguistic patterns, to precious commonsense knowledge defined in ConceptNet (Zhang et al., 2020b). Beyond that, we will also introduce how to automatically acquire other event-centric commonsense knowledge including but not limited to temporal properties (Zhou et al., 2020), intentions (Chen et al., 2020), effects (Sap et al., 2019) and graph schemas (Li et al., 2020c) of events. This part is estimated to be 35 minutes.

Event Summarization [30min]
In addition to specific, individual events, we are also interested in large-scale events that unfold over time. Over the past year, we saw many examples of such events, including COVID-19, the vaccine roll-out, the Black Lives Matter movement and the US presidential election. In this tutorial, we will present methods for tracking such events over time and generating summaries that provide updates as an event unfolds. The task of identifying and tracking events was first introduced in the Topic Detection and Tracking challenge (Allan et al., 1998). Recent work has explored new methods for tracking and visualizing such events over time (e.g., (Laban and Hearst, 2017;Miranda et al., 2018;Staykovski et al., 2019;Saravanakumar et al., 2021)), in some cases generating summaries that contain information on what is new (e.g., (Kedzie et al., 2015(Kedzie et al., , 2018) and in other cases, exploring timeline summarization, ordering events and generating summaries that are placed along a timeline (e.g., (Wang et al., 2015;Binh Tran et al., 2013;Chen et al., 2019;Nguyen et al., 2014)) We will also consider how these are related to summarization of an event that takes place within a single day, a problem that falls within the category of multidocument summarization (e.g., (Liu and Lapata, 2019;Fabbri et al., 2019)), as typically there may be many articles covering the same event. By using multiple articles as input, a summarizer can present different perspectives on the same event as well as identify salient information that is highlighted many in different ways across the set of input articles. This part is scheduled for 30 minutes

Emerging Research Problems [20min]
Event-centric NLP impacts on a wide spectrum of knowledge-driven AI tasks, and is particularly knotted with commonsense understanding. We will conclude the tutorial using 20 minutes by presenting some challenges and potential research topics in applying eventuality knowledge in downstream tasks (e.g., reading comprehension, dialogue generation, and event timeline generation), and grounding eventuality knowledge to visual modalities, and challenges for cross-document event consolidation with human-defined schemas.

Specification of the Tutorial
The proposed tutorial is considered a cuttingedge tutorial that introduces the recent advances in an emerging area of NLP. The presented topic has not been covered by previous ACL/EMNLP/NAACL/EACL/COLING tutorials in the past 4 years. This tutorial has not been presented elsewhere, while a more AI-flavored version with a subset of the contents has been planned in parallel at AAAI 2021, to be presented by a subset of the instructors. We estimate that at least 60% of the works covered in this tutorial are from researchers other than the instructors. Audience and Prerequisites While no specific background knowledge is assumed of the audience, it would be best for the attendees to know about basic deep learning technologies, pre-trained word embeddings (e.g. Word2Vec) and language models (e.g. BERT). The following is a reading list that could help provide background knowledge to the audience before attending this tutorial: • Emmon Bach. The algebra of events. Linguistics and philosophy. 9(1):5-16, 1986. Dan Roth is the Eduardo D. Glandt Distinguished Professor at the Department of Computer and Information Science, University of Pennsylvania, and a Fellow of the AAAS, ACM, AAAI, and the ACL.
In 2017 Roth was awarded the John McCarthy Award, the highest award the AI community gives to mid-career AI researchers. Roth was recognized "for major conceptual and theoretical advances in the modeling of natural language understanding, machine learning, and reasoning." Roth has published broadly in machine learning, natural language processing, knowledge representation and reasoning, and learning theory, and has developed advanced machine learning based tools for natural language applications that are being used widely. Roth has given tutorials on these and other topics in all ACL and AAAI major conferences. Until February 2017 Roth was the Editor-in-Chief of the Journal of Artificial Intelligence Research (JAIR). He was the program chair of AAAI'11, ACL'03 and CoNLL'02, and serves regularly as an area chair and senior program committee member in the major conferences in his research areas. Prof. Roth received his B.A Summa cum laude in Mathematics from the Technion, Israel, and his Ph.D. in Computer Science from Harvard University in 1995. Additional information is available at http://www.cis.upenn.edu/˜danroth/.