DISK-CSV: Distilling Interpretable Semantic Knowledge with a Class Semantic Vector

Housam Khalifa Bashier, Mi-Young Kim, Randy Goebel


Abstract
Neural networks (NN) applied to natural language processing (NLP) are becoming deeper and more complex, making them increasingly difficult to understand and interpret. Even in applications of limited scope on fixed data, the creation of these complex “black-boxes” creates substantial challenges for debugging, understanding, and generalization. But rapid development in this field has now lead to building more straightforward and interpretable models. We propose a new technique (DISK-CSV) to distill knowledge concurrently from any neural network architecture for text classification, captured as a lightweight interpretable/explainable classifier. Across multiple datasets, our approach achieves better performance than the target black-box. In addition, our approach provides better explanations than existing techniques.
Anthology ID:
2021.eacl-main.263
Volume:
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
Month:
April
Year:
2021
Address:
Online
Editors:
Paola Merlo, Jorg Tiedemann, Reut Tsarfaty
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3021–3030
Language:
URL:
https://aclanthology.org/2021.eacl-main.263
DOI:
10.18653/v1/2021.eacl-main.263
Bibkey:
Cite (ACL):
Housam Khalifa Bashier, Mi-Young Kim, and Randy Goebel. 2021. DISK-CSV: Distilling Interpretable Semantic Knowledge with a Class Semantic Vector. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3021–3030, Online. Association for Computational Linguistics.
Cite (Informal):
DISK-CSV: Distilling Interpretable Semantic Knowledge with a Class Semantic Vector (Bashier et al., EACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.eacl-main.263.pdf
Data
AG NewsIMDb Movie Reviews