John DeNero


2021

pdf bib
Interactive Assignments for Teaching Structured Neural NLP
David Gaddy | Daniel Fried | Nikita Kitaev | Mitchell Stern | Rodolfo Corona | John DeNero | Dan Klein
Proceedings of the Fifth Workshop on Teaching NLP

We present a set of assignments for a graduate-level NLP course. Assignments are designed to be interactive, easily gradable, and to give students hands-on experience with several key types of structure (sequences, tags, parse trees, and logical forms), modern neural architectures (LSTMs and Transformers), inference algorithms (dynamic programs and approximate search) and training methods (full and weak supervision). We designed assignments to build incrementally both within each assignment and across assignments, with the goal of enabling students to undertake graduate-level research in NLP by the end of the course.

2020

pdf bib
End-to-End Neural Word Alignment Outperforms GIZA++
Thomas Zenkel | Joern Wuebker | John DeNero
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Word alignment was once a core unsupervised learning task in natural language processing because of its essential role in training statistical machine translation (MT) models. Although unnecessary for training neural MT models, word alignment still plays an important role in interactive applications of neural machine translation, such as annotation transfer and lexicon injection. While statistical MT methods have been replaced by neural approaches with superior performance, the twenty-year-old GIZA++ toolkit remains a key component of state-of-the-art word alignment systems. Prior work on neural word alignment has only been able to outperform GIZA++ by using its output during training. We present the first end-to-end neural word alignment method that consistently outperforms GIZA++ on three data sets. Our approach repurposes a Transformer model trained for supervised translation to also serve as an unsupervised word alignment model in a manner that is tightly integrated and does not affect translation quality.

pdf bib
A Streaming Approach For Efficient Batched Beam Search
Kevin Yang | Violet Yao | John DeNero | Dan Klein
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

We propose an efficient batching strategy for variable-length decoding on GPU architectures. During decoding, when candidates terminate or are pruned according to heuristics, our streaming approach periodically “refills” the batch before proceeding with a selected subset of candidates. We apply our method to variable-width beam search on a state-of-the-art machine translation model. Our method decreases runtime by up to 71% compared to a fixed-width beam search baseline and 17% compared to a variable-width baseline, while matching baselines’ BLEU. Finally, experiments show that our method can speed up decoding in other domains, such as semantic and syntactic parsing.

2019

pdf bib
Measuring Immediate Adaptation Performance for Neural Machine Translation
Patrick Simianer | Joern Wuebker | John DeNero
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)

Incremental domain adaptation, in which a system learns from the correct output for each input immediately after making its prediction for that input, can dramatically improve system performance for interactive machine translation. Users of interactive systems are sensitive to the speed of adaptation and how often a system repeats mistakes, despite being corrected. Adaptation is most commonly assessed using corpus-level BLEU- or TER-derived metrics that do not explicitly take adaptation speed into account. We find that these metrics often do not capture immediate adaptation effects, such as zero-shot and one-shot learning of domain-specific lexical items. To this end, we propose new metrics that directly evaluate immediate adaptation performance for machine translation. We use these metrics to choose the most suitable adaptation method from a range of different adaptation techniques for neural machine translation systems.

2018

pdf bib
Compact Personalized Models for Neural Machine Translation
Joern Wuebker | Patrick Simianer | John DeNero
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

We propose and compare methods for gradient-based domain adaptation of self-attentive neural machine translation models. We demonstrate that a large proportion of model parameters can be frozen during adaptation with minimal or no reduction in translation quality by encouraging structured sparsity in the set of offset tensors during learning via group lasso regularization. We evaluate this technique for both batch and incremental adaptation across multiple data sets and language pairs. Our system architecture–combining a state-of-the-art self-attentive model with compact domain adaptation–provides high quality personalized machine translation that is both space and time efficient.

2016

pdf bib
Models and Inference for Prefix-Constrained Machine Translation
Joern Wuebker | Spence Green | John DeNero | Saša Hasan | Minh-Thang Luong
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

pdf bib
An Analysis of the Ability of Statistical Language Models to Capture the Structural Properties of Language
Aneiss Ghodsi | John DeNero
Proceedings of the 9th International Natural Language Generation conference

pdf bib
Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations
John DeNero | Mark Finlayson | Sravana Reddy
Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations

2015

pdf bib
Hierarchical Incremental Adaptation for Statistical Machine Translation
Joern Wuebker | Spence Green | John DeNero
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

pdf bib
Variable-Length Word Encodings for Neural Translation Models
Rohan Chitnis | John DeNero
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

2014

pdf bib
A Constrained Viterbi Relaxation for Bidirectional Word Alignment
Yin-Wen Chang | Alexander M. Rush | John DeNero | Michael Collins
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

pdf bib
Observational Initialization of Type-Supervised Taggers
Hui Zhang | John DeNero
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

2013

pdf bib
Identifying Phrasal Verbs Using Many Bilingual Corpora
Karl Pichotta | John DeNero
Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing

pdf bib
Supervised Learning of Complete Morphological Paradigms
Greg Durrett | John DeNero
Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

2012

pdf bib
Unsupervised Translation Sense Clustering
Mohit Bansal | John DeNero | Dekang Lin
Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

pdf bib
A Class-Based Agreement Model for Generating Accurately Inflected Translations
Spence Green | John DeNero
Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

pdf bib
A Feature-Rich Constituent Context Model for Grammar Induction
Dave Golland | John DeNero | Jakob Uszkoreit
Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

2011

pdf bib
Inducing Sentence Structure from Parallel Corpora for Reordering
John DeNero | Jakob Uszkoreit
Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing

pdf bib
Model-Based Aligner Combination Using Dual Decomposition
John DeNero | Klaus Macherey
Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies

2010

pdf bib
Painless Unsupervised Learning with Features
Taylor Berg-Kirkpatrick | Alexandre Bouchard-Côté | John DeNero | Dan Klein
Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics

pdf bib
Model Combination for Machine Translation
John DeNero | Shankar Kumar | Ciprian Chelba | Franz Och
Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics

pdf bib
Discriminative Modeling of Extraction Sets for Machine Translation
John DeNero | Dan Klein
Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics

2009

pdf bib
Fast Consensus Decoding over Translation Forests
John DeNero | David Chiang | Kevin Knight
Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP

pdf bib
Better Word Alignments with Supervised ITG Models
Aria Haghighi | John Blitzer | John DeNero | Dan Klein
Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP

pdf bib
Asynchronous Binarization for Synchronous Grammars
John DeNero | Adam Pauls | Dan Klein
Proceedings of the ACL-IJCNLP 2009 Conference Short Papers

pdf bib
Consensus Training for Consensus Decoding in Machine Translation
Adam Pauls | John DeNero | Dan Klein
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

pdf bib
Efficient Parsing for Transducer Grammars
John DeNero | Mohit Bansal | Adam Pauls | Dan Klein
Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics

2008

pdf bib
Sampling Alignment Structure under a Bayesian Translation Model
John DeNero | Alexandre Bouchard-Côté | Dan Klein
Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing

pdf bib
The Complexity of Phrase Alignment Problems
John DeNero | Dan Klein
Proceedings of ACL-08: HLT, Short Papers

2007

pdf bib
Approximate Factoring for A* Search
Aria Haghighi | John DeNero | Dan Klein
Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference

pdf bib
Tailoring Word Alignments to Syntactic Machine Translation
John DeNero | Dan Klein
Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics

2006

pdf bib
Why Generative Phrase Models Underperform Surface Heuristics
John DeNero | Dan Gillick | James Zhang | Dan Klein
Proceedings on the Workshop on Statistical Machine Translation