MPII: Multi-Level Mutual Promotion for Inference and Interpretation

Yan Liu, Sanyuan Chen, Yazheng Yang, Qi Dai


Abstract
In order to better understand the rationale behind model behavior, recent works have exploited providing interpretation to support the inference prediction. However, existing methods tend to provide human-unfriendly interpretation, and are prone to sub-optimal performance due to one-side promotion, i.e. either inference promotion with interpretation or vice versa. In this paper, we propose a multi-level Mutual Promotion mechanism for self-evolved Inference and sentence-level Interpretation (MPII). Specifically, from the model-level, we propose a Step-wise Integration Mechanism to jointly perform and deeply integrate inference and interpretation in an autoregressive manner. From the optimization-level, we propose an Adversarial Fidelity Regularization to improve the fidelity between inference and interpretation with the Adversarial Mutual Information training strategy. Extensive experiments on NLI and CQA tasks reveal that the proposed MPII approach can significantly outperform baseline models for both the inference performance and the interpretation quality.
Anthology ID:
2022.acl-long.488
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7074–7084
Language:
URL:
https://aclanthology.org/2022.acl-long.488
DOI:
10.18653/v1/2022.acl-long.488
Bibkey:
Cite (ACL):
Yan Liu, Sanyuan Chen, Yazheng Yang, and Qi Dai. 2022. MPII: Multi-Level Mutual Promotion for Inference and Interpretation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7074–7084, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
MPII: Multi-Level Mutual Promotion for Inference and Interpretation (Liu et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.488.pdf
Code
 thenamek/mpii
Data
CoS-EMultiNLISNLIe-SNLI