Annotating Evaluative Language: Challenges and Solutions in Applying Appraisal Theory

Jiamei Zeng, Min Dong, Alex Chengyu Fang


Abstract
This article describes a corpus-based experiment to identify the challenges and solutions in the annotation of evaluative language according to the scheme defined in Appraisal Theory (Martin and White, 2005). Originating from systemic functional linguistics, Appraisal Theory provides a robust framework for the analysis of linguistic expressions of evaluation, stance, and interpersonal relationships. Despite its theoretical richness, the practical application of Appraisal Theory in text annotation presents significant challenges, chiefly due to the intricacies of identifying and classifying evaluative expressions within its sub-system of Attitude, which comprises Affect, Judgement, and Appreciation. This study examines these challenges through the annotation of a corpus of editorials related to the Russian-Ukraine conflict and aims to offer practical solutions to enhance the transparency and consistency of the annotation. By refining the annotation process and addressing the subjective nature in the identification and classification of evaluative language, this work represents some timely effort in the annotation of pragmatic knowledge in language resources.
Anthology ID:
2024.isa-1.17
Volume:
Proceedings of the 20th Joint ACL - ISO Workshop on Interoperable Semantic Annotation @ LREC-COLING 2024
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Harry Bunt, Nancy Ide, Kiyong Lee, Volha Petukhova, James Pustejovsky, Laurent Romary
Venues:
ISA | WS
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
144–151
Language:
URL:
https://aclanthology.org/2024.isa-1.17
DOI:
Bibkey:
Cite (ACL):
Jiamei Zeng, Min Dong, and Alex Chengyu Fang. 2024. Annotating Evaluative Language: Challenges and Solutions in Applying Appraisal Theory. In Proceedings of the 20th Joint ACL - ISO Workshop on Interoperable Semantic Annotation @ LREC-COLING 2024, pages 144–151, Torino, Italia. ELRA and ICCL.
Cite (Informal):
Annotating Evaluative Language: Challenges and Solutions in Applying Appraisal Theory (Zeng et al., ISA-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.isa-1.17.pdf