%0 Conference Proceedings %T Does it Make Sense? And Why? A Pilot Study for Sense Making and Explanation %A Wang, Cunxiang %A Liang, Shuailong %A Zhang, Yue %A Li, Xiaonan %A Gao, Tian %Y Korhonen, Anna %Y Traum, David %Y Màrquez, Lluís %S Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics %D 2019 %8 July %I Association for Computational Linguistics %C Florence, Italy %F wang-etal-2019-make %X Introducing common sense to natural language understanding systems has received increasing research attention. It remains a fundamental question on how to evaluate whether a system has the sense-making capability. Existing benchmarks measure common sense knowledge indirectly or without reasoning. In this paper, we release a benchmark to directly test whether a system can differentiate natural language statements that make sense from those that do not make sense. In addition, a system is asked to identify the most crucial reason why a statement does not make sense. We evaluate models trained over large-scale language modeling tasks as well as human performance, showing that there are different challenges for system sense-making. %R 10.18653/v1/P19-1393 %U https://aclanthology.org/P19-1393 %U https://doi.org/10.18653/v1/P19-1393 %P 4020-4026