A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering

Zhihong Shao, Lifeng Shang, Qun Liu, Minlie Huang


Abstract
Weakly supervised question answering usually has only the final answers as supervision signals while the correct solutions to derive the answers are not provided. This setting gives rise to the spurious solution problem: there may exist many spurious solutions that coincidentally derive the correct answer, but training on such solutions can hurt model performance (e.g., producing wrong solutions or answers). For example, for discrete reasoning tasks as on DROP, there may exist many equations to derive a numeric answer, and typically only one of them is correct. Previous learning methods mostly filter out spurious solutions with heuristics or using model confidence, but do not explicitly exploit the semantic correlations between a question and its solution. In this paper, to alleviate the spurious solution problem, we propose to explicitly exploit such semantic correlations by maximizing the mutual information between question-answer pairs and predicted solutions. Extensive experiments on four question answering datasets show that our method significantly outperforms previous learning methods in terms of task performance and is more effective in training models to produce correct solutions.
Anthology ID:
2021.acl-long.318
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4111–4124
Language:
URL:
https://aclanthology.org/2021.acl-long.318
DOI:
10.18653/v1/2021.acl-long.318
Bibkey:
Cite (ACL):
Zhihong Shao, Lifeng Shang, Qun Liu, and Minlie Huang. 2021. A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4111–4124, Online. Association for Computational Linguistics.
Cite (Informal):
A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering (Shao et al., ACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-long.318.pdf
Video:
 https://aclanthology.org/2021.acl-long.318.mp4
Code
 ZhihongShao/MIMAX
Data
DROPQUASARQUASAR-TWebQuestionsWikiSQL