Exact Hard Monotonic Attention for Character-Level Transduction

Shijie Wu, Ryan Cotterell


Abstract
Many common character-level, string-to-string transduction tasks, e.g., grapheme-to-phoneme conversion and morphological inflection, consist almost exclusively of monotonic transduction. Neural sequence-to-sequence models with soft attention, non-monotonic models, outperform popular monotonic models. In this work, we ask the following question: Is monotonicity really a helpful inductive bias in these tasks? We develop a hard attention sequence-to-sequence model that enforces strict monotonicity and learns alignment jointly. With the help of dynamic programming, we are able to compute the exact marginalization over all alignments. Our models achieve state-of-the-art performance on morphological inflection. Furthermore, we find strong performance on two other character-level transduction tasks. Code is available at https://github.com/shijie-wu/neural-transducer.
Anthology ID:
P19-1148
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1530–1537
Language:
URL:
https://aclanthology.org/P19-1148
DOI:
10.18653/v1/P19-1148
Bibkey:
Cite (ACL):
Shijie Wu and Ryan Cotterell. 2019. Exact Hard Monotonic Attention for Character-Level Transduction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1530–1537, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Exact Hard Monotonic Attention for Character-Level Transduction (Wu & Cotterell, ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1148.pdf
Code
 shijie-wu/neural-transducer +  additional community code