Neural Math Word Problem Solver with Reinforcement Learning

Danqing Huang, Jing Liu, Chin-Yew Lin, Jian Yin


Abstract
Sequence-to-sequence model has been applied to solve math word problems. The model takes math problem descriptions as input and generates equations as output. The advantage of sequence-to-sequence model requires no feature engineering and can generate equations that do not exist in training data. However, our experimental analysis reveals that this model suffers from two shortcomings: (1) generate spurious numbers; (2) generate numbers at wrong positions. In this paper, we propose incorporating copy and alignment mechanism to the sequence-to-sequence model (namely CASS) to address these shortcomings. To train our model, we apply reinforcement learning to directly optimize the solution accuracy. It overcomes the “train-test discrepancy” issue of maximum likelihood estimation, which uses the surrogate objective of maximizing equation likelihood during training while the evaluation metric is solution accuracy (non-differentiable) at test time. Furthermore, to explore the effectiveness of our neural model, we use our model output as a feature and incorporate it into the feature-based model. Experimental results show that (1) The copy and alignment mechanism is effective to address the two issues; (2) Reinforcement learning leads to better performance than maximum likelihood on this task; (3) Our neural model is complementary to the feature-based model and their combination significantly outperforms the state-of-the-art results.
Anthology ID:
C18-1018
Volume:
Proceedings of the 27th International Conference on Computational Linguistics
Month:
August
Year:
2018
Address:
Santa Fe, New Mexico, USA
Editors:
Emily M. Bender, Leon Derczynski, Pierre Isabelle
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
213–223
Language:
URL:
https://aclanthology.org/C18-1018
DOI:
Bibkey:
Cite (ACL):
Danqing Huang, Jing Liu, Chin-Yew Lin, and Jian Yin. 2018. Neural Math Word Problem Solver with Reinforcement Learning. In Proceedings of the 27th International Conference on Computational Linguistics, pages 213–223, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Cite (Informal):
Neural Math Word Problem Solver with Reinforcement Learning (Huang et al., COLING 2018)
Copy Citation:
PDF:
https://aclanthology.org/C18-1018.pdf
Data
ALG514