Overconfidence in the Face of Ambiguity with Adversarial Data

Margaret Li, Julian Michael


Abstract
Adversarial data collection has shown promise as a method for building models which are more robust to the spurious correlations that generally appear in naturalistic data. However, adversarially-collected data may itself be subject to biases, particularly with regard to ambiguous or arguable labeling judgments. Searching for examples where an annotator disagrees with a model might over-sample ambiguous inputs, and filtering the results for high inter-annotator agreement may under-sample them. In either case, training a model on such data may produce predictable and unwanted biases. In this work, we investigate whether models trained on adversarially-collected data are miscalibrated with respect to the ambiguity of their inputs. Using Natural Language Inference models as a testbed, we find no clear difference in accuracy between naturalistically and adversarially trained models, but our model trained only on adversarially-sourced data is considerably more overconfident of its predictions and demonstrates worse calibration, especially on ambiguous inputs. This effect is mitigated, however, when naturalistic and adversarial training data are combined.
Anthology ID:
2022.dadc-1.4
Volume:
Proceedings of the First Workshop on Dynamic Adversarial Data Collection
Month:
July
Year:
2022
Address:
Seattle, WA
Venues:
DADC | NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
30–40
Language:
URL:
https://aclanthology.org/2022.dadc-1.4
DOI:
10.18653/v1/2022.dadc-1.4
Bibkey:
Cite (ACL):
Margaret Li and Julian Michael. 2022. Overconfidence in the Face of Ambiguity with Adversarial Data. In Proceedings of the First Workshop on Dynamic Adversarial Data Collection, pages 30–40, Seattle, WA. Association for Computational Linguistics.
Cite (Informal):
Overconfidence in the Face of Ambiguity with Adversarial Data (Li & Michael, DADC 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.dadc-1.4.pdf
Code
 julianmichael/aeae
Data
ANLIChaosNLIGLUEMultiNLI