Yunjia Qi
2024
ADELIE: Aligning Large Language Models on Information Extraction
Yunjia Qi
|
Hao Peng
|
Xiaozhi Wang
|
Bin Xu
|
Lei Hou
|
Juanzi Li
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
MAVEN-FACT: A Large-scale Event Factuality Detection Dataset
Chunyang Li
|
Hao Peng
|
Xiaozhi Wang
|
Yunjia Qi
|
Lei Hou
|
Bin Xu
|
Juanzi Li
Findings of the Association for Computational Linguistics: EMNLP 2024
Event Factuality Detection (EFD) task determines the factuality of textual events, i.e., classifying whether an event is a fact, possibility, or impossibility, which is essential for faithfully understanding and utilizing event knowledge. However, due to the lack of high-quality large-scale data, event factuality detection is under-explored in event understanding research, which limits the development of EFD community. To address these issues and provide faithful event understanding, we introduce MAVEN-FACT, a large-scale and high-quality EFD dataset based on the MAVEN dataset. MAVEN-FACT includes factuality annotations of 112,276 events, making it the largest EFD dataset. Extensive experiments demonstrate that MAVEN-FACT is challenging for both conventional fine-tuned models and large language models (LLMs). Thanks to the comprehensive annotations of event arguments and relations in MAVEN, MAVEN-FACT also supports some further analyses and we find that adopting event arguments and relations helps in event factuality detection for fine-tuned models but does not benefit LLMs. Furthermore, we preliminarily study an application case of event factuality detection and find it helps in mitigating event-related hallucination in LLMs. We will release our dataset and codes to facilitate further research on event factuality detection.