Hakusen Shu
2025
ML-Promise: A Multilingual Dataset for Corporate Promise Verification
Yohei Seki
|
Hakusen Shu
|
Anaïs Lhuissier
|
Hanwool Lee
|
Juyeon Kang
|
Min-Yuh Day
|
Chung-Chi Chen
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Promises made by politicians, corporate leaders, and public figures have a significant impact on public perception, trust, and institutional reputation. However, the complexity and volume of such commitments, coupled with difficulties in verifying their fulfillment, necessitate innovative methods for assessing their credibility. This paper introduces the concept of Promise Verification, a systematic approach involving steps such as promise identification, evidence assessment, and the evaluation of timing for verification. We propose the first multilingual dataset, ML-Promise, which includes English, French, Chinese, Japanese, and Korean, aimed at facilitating in-depth verification of promises, particularly in the context of Environmental, Social, and Governance (ESG) reports. Given the growing emphasis on corporate environmental contributions, this dataset addresses the challenge of evaluating corporate promises, especially in light of practices like greenwashing. Our findings also explore textual and image-based baselines, with promising results from retrieval-augmented generation (RAG) approaches. This work aims to foster further discourse on the accountability of public commitments across multiple languages and domains.
SemEval-2025 Task 6: Multinational, Multilingual, Multi-Industry Promise Verification
Chung-Chi Chen
|
Yohei Seki
|
Hakusen Shu
|
Anaïs Lhuissier
|
Juyeon Kang
|
Hanwool Lee
|
Min-Yuh Day
|
Hiroya Takamura
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
While extensive research exists on misinformation and disinformation, there is limited focus on future-oriented commitments, such as corporate ESG promises, which are often difficult to verify yet significantly impact public trust and market stability. To address this gap, we introduce the task of promise verification, leveraging natural language processing (NLP) techniques to automatically detect ESG commitments, identify supporting evidence, and evaluate the consistency between promises and evidence, while also inferring potential verification time points. This paper presents the dataset used in SemEval-2025 PromiseEval, outlines participant solutions, and discusses key findings. The goal is to enhance transparency in corporate discourse, strengthen investor trust, and support regulators in monitoring the fulfillment of corporate commitments.
Search
Fix author
Co-authors
- Chung-Chi Chen 2
- Min-Yuh Day 2
- Juyeon Kang 2
- Hanwool Lee 2
- Anaïs Lhuissier 2
- show all...