Structured Self-AttentionWeights Encode Semantics in Sentiment Analysis

Zhengxuan Wu, Thanh-Son Nguyen, Desmond Ong


Abstract
Neural attention, especially the self-attention made popular by the Transformer, has become the workhorse of state-of-the-art natural language processing (NLP) models. Very recent work suggests that the self-attention in the Transformer encodes syntactic information; Here, we show that self-attention scores encode semantics by considering sentiment analysis tasks. In contrast to gradient-based feature attribution methods, we propose a simple and effective Layer-wise Attention Tracing (LAT) method to analyze structured attention weights. We apply our method to Transformer models trained on two tasks that have surface dissimilarities, but share common semantics—sentiment analysis of movie reviews and time-series valence prediction in life story narratives. Across both tasks, words with high aggregated attention weights were rich in emotional semantics, as quantitatively validated by an emotion lexicon labeled by human annotators. Our results show that structured attention weights encode rich semantics in sentiment analysis, and match human interpretations of semantics.
Anthology ID:
2020.blackboxnlp-1.24
Volume:
Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
Month:
November
Year:
2020
Address:
Online
Editors:
Afra Alishahi, Yonatan Belinkov, Grzegorz Chrupała, Dieuwke Hupkes, Yuval Pinter, Hassan Sajjad
Venue:
BlackboxNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
255–264
Language:
URL:
https://aclanthology.org/2020.blackboxnlp-1.24
DOI:
10.18653/v1/2020.blackboxnlp-1.24
Bibkey:
Cite (ACL):
Zhengxuan Wu, Thanh-Son Nguyen, and Desmond Ong. 2020. Structured Self-AttentionWeights Encode Semantics in Sentiment Analysis. In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 255–264, Online. Association for Computational Linguistics.
Cite (Informal):
Structured Self-AttentionWeights Encode Semantics in Sentiment Analysis (Wu et al., BlackboxNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.blackboxnlp-1.24.pdf
Optional supplementary material:
 2020.blackboxnlp-1.24.OptionalSupplementaryMaterial.zip
Code
 frankaging/LAT_for_Transformer
Data
SENDSSTSST-5