Transformers-Based Approach for a Sustainability Term-Based Sentiment Analysis (STBSA)

Blaise Sandwidi, Suneer Pallitharammal Mukkolakal


Abstract
Traditional sentiment analysis is a sentence level or document-level task. However, a sentence or paragraph may contain multiple target terms with different sentiments, making sentiment prediction more challenging. Although pre-trained language models like BERT have been successful, incorporating dynamic semantic changes into aspect-based sentiment models remains difficult, especially for domain-specific sentiment analysis. To this end, in this paper, we propose a Term-Based Sentiment Analysis (TBSA), a novel method designed to learn Environmental, Social, and Governance (ESG) contexts based on a sustainability taxonomy for ESG aspect-oriented sentiment analysis. Notably, we introduce a technique enhancing the ESG term’s attention, inspired by the success of attention-based neural networks in machine translation (Bahdanau et al., 2015) and Computer Vision (Bello et al., 2019). It enables the proposed model to focus on a small region of the sentences at each step and to reweigh the crucial terms for a better understanding of the ESG aspect-aware sentiment. Beyond the novelty in the model design, we propose a new dataset of 125,000+ ESG analyst annotated data points for sustainability term based sentiment classification, which derives from historical sustainability corpus data and expertise acquired by development finance institutions. Our extensive experiments combining the new method and the new dataset demonstrate the effectiveness of the Sustainability TBSA model with an accuracy of 91.30% (90% F1-score). Both internal and external business applications of our model show an evident potential for a significant positive impact toward furthering sustainable development goals (SDGs).
Anthology ID:
2022.nlp4pi-1.19
Volume:
Proceedings of the Second Workshop on NLP for Positive Impact (NLP4PI)
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates (Hybrid)
Editors:
Laura Biester, Dorottya Demszky, Zhijing Jin, Mrinmaya Sachan, Joel Tetreault, Steven Wilson, Lu Xiao, Jieyu Zhao
Venue:
NLP4PI
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
157–170
Language:
URL:
https://aclanthology.org/2022.nlp4pi-1.19
DOI:
10.18653/v1/2022.nlp4pi-1.19
Bibkey:
Cite (ACL):
Blaise Sandwidi and Suneer Pallitharammal Mukkolakal. 2022. Transformers-Based Approach for a Sustainability Term-Based Sentiment Analysis (STBSA). In Proceedings of the Second Workshop on NLP for Positive Impact (NLP4PI), pages 157–170, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Cite (Informal):
Transformers-Based Approach for a Sustainability Term-Based Sentiment Analysis (STBSA) (Sandwidi & Pallitharammal Mukkolakal, NLP4PI 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.nlp4pi-1.19.pdf