On Committee Representations of Adversarial Learning Models for Question-Answer Ranking

Sparsh Gupta, Vitor Carvalho


Abstract
Adversarial training is a process in Machine Learning that explicitly trains models on adversarial inputs (inputs designed to deceive or trick the learning process) in order to make it more robust or accurate. In this paper we investigate how representing adversarial training models as committees can be used to effectively improve the performance of Question-Answer (QA) Ranking. We start by empirically probing the effects of adversarial training over multiple QA ranking algorithms, including the state-of-the-art Multihop Attention Network model. We evaluate these algorithms on several benchmark datasets and observe that, while adversarial training is beneficial to most baseline algorithms, there are cases where it may lead to overfitting and performance degradation. We investigate the causes of such degradation, and then propose a new representation procedure for this adversarial learning problem, based on committee learning, that not only is capable of consistently improving all baseline algorithms, but also outperforms the previous state-of-the-art algorithm by as much as 6% in NDCG.
Anthology ID:
W19-4325
Volume:
Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
Month:
August
Year:
2019
Address:
Florence, Italy
Venues:
ACL | RepL4NLP | WS
SIG:
SIGREP
Publisher:
Association for Computational Linguistics
Note:
Pages:
218–223
Language:
URL:
https://aclanthology.org/W19-4325
DOI:
10.18653/v1/W19-4325
Bibkey:
Cite (ACL):
Sparsh Gupta and Vitor Carvalho. 2019. On Committee Representations of Adversarial Learning Models for Question-Answer Ranking. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 218–223, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
On Committee Representations of Adversarial Learning Models for Question-Answer Ranking (Gupta & Carvalho, 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-4325.pdf
Data
InsuranceQAWikiQA