Chitwan Saharia
2023
Character-Aware Models Improve Visual Text Rendering
Rosanne Liu
|
Dan Garrette
|
Chitwan Saharia
|
William Chan
|
Adam Roberts
|
Sharan Narang
|
Irina Blok
|
Rj Mical
|
Mohammad Norouzi
|
Noah Constant
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Current image generation models struggle to reliably produce well-formed visual text. In this paper, we investigate a key contributing factor: popular text-to-image models lack character-level input features, making it much harder to predict a word’s visual makeup as a series of glyphs. To quantify this effect, we conduct a series of experiments comparing character-aware vs. character-blind text encoders. In the text-only domain, we find that character-aware models provide large gains on a novel spelling task (WikiSpell). Applying our learnings to the visual domain, we train a suite of image generation models, and show that character-aware variants outperform their character-blind counterparts across a range of novel text rendering tasks (our DrawText benchmark). Our models set a much higher state-of-the-art on visual spelling, with 30+ point accuracy gains over competitors on rare words, despite training on far fewer examples.
2020
Non-Autoregressive Machine Translation with Latent Alignments
Chitwan Saharia
|
William Chan
|
Saurabh Saxena
|
Mohammad Norouzi
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
This paper presents two strong methods, CTC and Imputer, for non-autoregressive machine translation that model latent alignments with dynamic programming. We revisit CTC for machine translation and demonstrate that a simple CTC model can achieve state-of-the-art for single-step non-autoregressive machine translation, contrary to what prior work indicates. In addition, we adapt the Imputer model for non-autoregressive machine translation and demonstrate that Imputer with just 4 generation steps can match the performance of an autoregressive Transformer baseline. Our latent alignment models are simpler than many existing non-autoregressive translation baselines; for example, we do not require target length prediction or re-scoring with an autoregressive model. On the competitive WMT’14 En→De task, our CTC model achieves 25.7 BLEU with a single generation step, while Imputer achieves 27.5 BLEU with 2 generation steps, and 28.0 BLEU with 4 generation steps. This compares favourably to the autoregressive Transformer baseline at 27.8 BLEU.
Search
Co-authors
- William Chan 2
- Mohammad Norouzi 2
- Rosanne Liu 1
- Dan Garrette 1
- Adam Roberts 1
- show all...