Learning Constraints for Structured Prediction Using Rectifier Networks

Xingyuan Pan, Maitrey Mehta, Vivek Srikumar


Abstract
Various natural language processing tasks are structured prediction problems where outputs are constructed with multiple interdependent decisions. Past work has shown that domain knowledge, framed as constraints over the output space, can help improve predictive accuracy. However, designing good constraints often relies on domain expertise. In this paper, we study the problem of learning such constraints. We frame the problem as that of training a two-layer rectifier network to identify valid structures or substructures, and show a construction for converting a trained network into a system of linear constraints over the inference variables. Our experiments on several NLP tasks show that the learned constraints can improve the prediction accuracy, especially when the number of training examples is small.
Anthology ID:
2020.acl-main.438
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4843–4858
Language:
URL:
https://aclanthology.org/2020.acl-main.438
DOI:
10.18653/v1/2020.acl-main.438
Bibkey:
Cite (ACL):
Xingyuan Pan, Maitrey Mehta, and Vivek Srikumar. 2020. Learning Constraints for Structured Prediction Using Rectifier Networks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4843–4858, Online. Association for Computational Linguistics.
Cite (Informal):
Learning Constraints for Structured Prediction Using Rectifier Networks (Pan et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.438.pdf
Video:
 http://slideslive.com/38928961
Code
 utahnlp/learning-constraints