Controlled tasks for model analysis: Retrieving discrete information from sequences

Ionut-Teodor Sorodoc, Gemma Boleda, Marco Baroni


Abstract
In recent years, the NLP community has shown increasing interest in analysing how deep learning models work. Given that large models trained on complex tasks are difficult to inspect, some of this work has focused on controlled tasks that emulate specific aspects of language. We propose a new set of such controlled tasks to explore a crucial aspect of natural language processing that has not received enough attention: the need to retrieve discrete information from sequences. We also study model behavior on the tasks with simple instantiations of Transformers and LSTMs. Our results highlight the beneficial role of decoder attention and its sometimes unexpected interaction with other components. Moreover, we show that, for most of the tasks, these simple models still show significant difficulties. We hope that the community will take up the analysis possibilities that our tasks afford, and that a clearer understanding of model behavior on the tasks will lead to better and more transparent models.
Anthology ID:
2021.blackboxnlp-1.37
Volume:
Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Jasmijn Bastings, Yonatan Belinkov, Emmanuel Dupoux, Mario Giulianelli, Dieuwke Hupkes, Yuval Pinter, Hassan Sajjad
Venue:
BlackboxNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
468–478
Language:
URL:
https://aclanthology.org/2021.blackboxnlp-1.37
DOI:
10.18653/v1/2021.blackboxnlp-1.37
Bibkey:
Cite (ACL):
Ionut-Teodor Sorodoc, Gemma Boleda, and Marco Baroni. 2021. Controlled tasks for model analysis: Retrieving discrete information from sequences. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 468–478, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Controlled tasks for model analysis: Retrieving discrete information from sequences (Sorodoc et al., BlackboxNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.blackboxnlp-1.37.pdf
Code
 sorodoc/discreteseq