Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods

Nils Feldhus, Leonhard Hennig, Maximilian Dustin Nasert, Christopher Ebert, Robert Schwarzenberg, Sebastian Möller


Abstract
Saliency maps can explain a neural model’s predictions by identifying important input features. They are difficult to interpret for laypeople, especially for instances with many features. In order to make them more accessible, we formalize the underexplored task of translating saliency maps into natural language and compare methods that address two key challenges of this approach – what and how to verbalize. In both automatic and human evaluation setups, using token-level attributions from text classification tasks, we compare two novel methods (search-based and instruction-based verbalizations) against conventional feature importance representations (heatmap visualizations and extractive rationales), measuring simulatability, faithfulness, helpfulness and ease of understanding. Instructing GPT-3.5 to generate saliency map verbalizations yields plausible explanations which include associations, abstractive summarization and commonsense reasoning, achieving by far the highest human ratings, but they are not faithfully capturing numeric information and are inconsistent in their interpretation of the task. In comparison, our search-based, model-free verbalization approach efficiently completes templated verbalizations, is faithful by design, but falls short in helpfulness and simulatability. Our results suggest that saliency map verbalization makes feature attribution explanations more comprehensible and less cognitively challenging to humans than conventional representations.
Anthology ID:
2023.nlrse-1.4
Volume:
Proceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE)
Month:
June
Year:
2023
Address:
Toronto, Canada
Editors:
Bhavana Dalvi Mishra, Greg Durrett, Peter Jansen, Danilo Neves Ribeiro, Jason Wei
Venue:
NLRSE
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
30–46
Language:
URL:
https://aclanthology.org/2023.nlrse-1.4
DOI:
10.18653/v1/2023.nlrse-1.4
Bibkey:
Cite (ACL):
Nils Feldhus, Leonhard Hennig, Maximilian Dustin Nasert, Christopher Ebert, Robert Schwarzenberg, and Sebastian Möller. 2023. Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods. In Proceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE), pages 30–46, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods (Feldhus et al., NLRSE 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.nlrse-1.4.pdf