%0 Conference Proceedings %T Mapping natural language commands to web elements %A Pasupat, Panupong %A Jiang, Tian-Shun %A Liu, Evan %A Guu, Kelvin %A Liang, Percy %Y Riloff, Ellen %Y Chiang, David %Y Hockenmaier, Julia %Y Tsujii, Jun’ichi %S Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing %D 2018 %8 oct nov %I Association for Computational Linguistics %C Brussels, Belgium %F pasupat-etal-2018-mapping %X The web provides a rich, open-domain environment with textual, structural, and spatial properties. We propose a new task for grounding language in this environment: given a natural language command (e.g., “click on the second article”), choose the correct element on the web page (e.g., a hyperlink or text box). We collected a dataset of over 50,000 commands that capture various phenomena such as functional references (e.g. “find who made this site”), relational reasoning (e.g. “article by john”), and visual reasoning (e.g. “top-most article”). We also implemented and analyzed three baseline models that capture different phenomena present in the dataset. %R 10.18653/v1/D18-1540 %U https://aclanthology.org/D18-1540 %U https://doi.org/10.18653/v1/D18-1540 %P 4970-4976