Modular Networks for Compositional Instruction Following

Rodolfo Corona, Daniel Fried, Coline Devin, Dan Klein, Trevor Darrell


Abstract
Standard architectures used in instruction following often struggle on novel compositions of subgoals (e.g. navigating to landmarks or picking up objects) observed during training. We propose a modular architecture for following natural language instructions that describe sequences of diverse subgoals. In our approach, subgoal modules each carry out natural language instructions for a specific subgoal type. A sequence of modules to execute is chosen by learning to segment the instructions and predicting a subgoal type for each segment. When compared to standard, non-modular sequence-to-sequence approaches on ALFRED, a challenging instruction following benchmark, we find that modularization improves generalization to novel subgoal compositions, as well as to environments unseen in training.
Anthology ID:
2021.naacl-main.81
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1033–1040
Language:
URL:
https://aclanthology.org/2021.naacl-main.81
DOI:
10.18653/v1/2021.naacl-main.81
Bibkey:
Cite (ACL):
Rodolfo Corona, Daniel Fried, Coline Devin, Dan Klein, and Trevor Darrell. 2021. Modular Networks for Compositional Instruction Following. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1033–1040, Online. Association for Computational Linguistics.
Cite (Informal):
Modular Networks for Compositional Instruction Following (Corona et al., NAACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-main.81.pdf
Video:
 https://aclanthology.org/2021.naacl-main.81.mp4
Data
ALFRED