Ensemble of MRR and NDCG models for Visual Dialog

Idan Schwartz


Abstract
Assessing an AI agent that can converse in human language and understand visual content is challenging. Generation metrics, such as BLEU scores favor correct syntax over semantics. Hence a discriminative approach is often used, where an agent ranks a set of candidate options. The mean reciprocal rank (MRR) metric evaluates the model performance by taking into account the rank of a single human-derived answer. This approach, however, raises a new challenge: the ambiguity and synonymy of answers, for instance, semantic equivalence (e.g., ‘yeah’ and ‘yes’). To address this, the normalized discounted cumulative gain (NDCG) metric has been used to capture the relevance of all the correct answers via dense annotations. However, the NDCG metric favors the usually applicable uncertain answers such as ‘I don’t know.’ Crafting a model that excels on both MRR and NDCG metrics is challenging. Ideally, an AI agent should answer a human-like reply and validate the correctness of any answer. To address this issue, we describe a two-step non-parametric ranking approach that can merge strong MRR and NDCG models. Using our approach, we manage to keep most MRR state-of-the-art performance (70.41% vs. 71.24%) and the NDCG state-of-the-art performance (72.16% vs. 75.35%). Moreover, our approach won the recent Visual Dialog 2020 challenge. Source code is available at https://github.com/idansc/mrr-ndcg.
Anthology ID:
2021.naacl-main.262
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Editors:
Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3272–3363
Language:
URL:
https://aclanthology.org/2021.naacl-main.262
DOI:
10.18653/v1/2021.naacl-main.262
Bibkey:
Cite (ACL):
Idan Schwartz. 2021. Ensemble of MRR and NDCG models for Visual Dialog. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3272–3363, Online. Association for Computational Linguistics.
Cite (Informal):
Ensemble of MRR and NDCG models for Visual Dialog (Schwartz, NAACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-main.262.pdf
Video:
 https://aclanthology.org/2021.naacl-main.262.mp4
Code
 idansc/mrr-ndcg
Data
VisDial