Markus Dreyer


2021

pdf bib
Rewards with Negative Examples for Reinforced Topic-Focused Abstractive Summarization
Khalil Mrini | Can Liu | Markus Dreyer
Proceedings of the Third Workshop on New Frontiers in Summarization

We consider the problem of topic-focused abstractive summarization, where the goal is to generate an abstractive summary focused on a particular topic, a phrase of one or multiple words. We hypothesize that the task of generating topic-focused summaries can be improved by showing the model what it must not focus on. We introduce a deep reinforcement learning approach to topic-focused abstractive summarization, trained on rewards with a novel negative example baseline. We define the input in this problem as the source text preceded by the topic. We adapt the CNN-Daily Mail and New York Times summarization datasets for this task. We then show through experiments on existing rewards that the use of a negative example baseline can outperform the use of a self-critical baseline, in Rouge, BERTScore, and human evaluation metrics.

pdf bib
Efficiently Summarizing Text and Graph Encodings of Multi-Document Clusters
Ramakanth Pasunuru | Mengwen Liu | Mohit Bansal | Sujith Ravi | Markus Dreyer
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

This paper presents an efficient graph-enhanced approach to multi-document summarization (MDS) with an encoder-decoder Transformer model. This model is based on recent advances in pre-training both encoder and decoder on very large text data (Lewis et al., 2019), and it incorporates an efficient encoding mechanism (Beltagy et al., 2020) that avoids the quadratic memory growth typical for traditional Transformers. We show that this powerful combination not only scales to large input documents commonly found when summarizing news clusters; it also enables us to process additional input in the form of auxiliary graph representations, which we derive from the multi-document clusters. We present a mechanism to incorporate such graph information into the encoder-decoder model that was pre-trained on text only. Our approach leads to significant improvements on the Multi-News dataset, overall leading to an average 1.8 ROUGE score improvement over previous work (Li et al., 2020). We also show improvements in a transfer-only setup on the DUC-2004 dataset. The graph encodings lead to summaries that are more abstractive. Human evaluation shows that they are also more informative and factually more consistent with their input documents.

2019

pdf bib
Multi-Task Networks with Universe, Group, and Task Feature Learning
Shiva Pentyala | Mengwen Liu | Markus Dreyer
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

We present methods for multi-task learning that take advantage of natural groupings of related tasks. Task groups may be defined along known properties of the tasks, such as task domain or language. Such task groups represent supervised information at the inter-task level and can be encoded into the model. We investigate two variants of neural network architectures that accomplish this, learning different feature spaces at the levels of individual tasks, task groups, as well as the universe of all tasks: (1) parallel architectures encode each input simultaneously into feature spaces at different levels; (2) serial architectures encode each input successively into feature spaces at different levels in the task hierarchy. We demonstrate the methods on natural language understanding (NLU) tasks, where a grouping of tasks into different task domains leads to improved performance on ATIS, Snips, and a large in-house dataset.

2017

pdf bib
Transfer Learning for Neural Semantic Parsing
Xing Fan | Emilio Monti | Lambert Mathias | Markus Dreyer
Proceedings of the 2nd Workshop on Representation Learning for NLP

The goal of semantic parsing is to map natural language to a machine interpretable meaning representation language (MRL). One of the constraints that limits full exploration of deep learning technologies for semantic parsing is the lack of sufficient annotation training data. In this paper, we propose using sequence-to-sequence in a multi-task setup for semantic parsing with focus on transfer learning. We explore three multi-task architectures for sequence-to-sequence model and compare their performance with the independently trained model. Our experiments show that the multi-task setup aids transfer learning from an auxiliary task with large labeled data to the target task with smaller labeled data. We see an absolute accuracy gain ranging from 1.0% to 4.4% in in our in-house data set and we also see good gains ranging from 2.5% to 7.0% on the ATIS semantic parsing tasks with syntactic and semantic auxiliary tasks.

2015

pdf bib
APRO: All-Pairs Ranking Optimization for MT Tuning
Markus Dreyer | Yuanzhe Dong
Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

pdf bib
hyp: A Toolkit for Representing, Manipulating, and Optimizing Hypergraphs
Markus Dreyer | Jonathan Graehl
Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations

2012

pdf bib
HyTER: Meaning-Equivalent Semantics for Translation Evaluation
Markus Dreyer | Daniel Marcu
Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

2011

pdf bib
Discovering Morphological Paradigms from Plain Text Using a Dirichlet Process Mixture Model
Markus Dreyer | Jason Eisner
Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing

2009

pdf bib
Graphical Models over Multiple Strings
Markus Dreyer | Jason Eisner
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

2008

pdf bib
Machine Translation System Combination using ITG-based Alignments
Damianos Karakos | Jason Eisner | Sanjeev Khudanpur | Markus Dreyer
Proceedings of ACL-08: HLT, Short Papers

pdf bib
Latent-Variable Modeling of String Transductions with Finite-State Methods
Markus Dreyer | Jason Smith | Jason Eisner
Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing

2007

pdf bib
Comparing Reordering Constraints for SMT Using Efficient BLEU Oracle Computation
Markus Dreyer | Keith Hall | Sanjeev Khudanpur
Proceedings of SSST, NAACL-HLT 2007 / AMTA Workshop on Syntax and Structure in Statistical Translation

2006

pdf bib
Better Informed Training of Latent Syntactic Features
Markus Dreyer | Jason Eisner
Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing

pdf bib
Vine Parsing and Minimum Risk Reranking for Speed and Precision
Markus Dreyer | David A. Smith | Noah A. Smith
Proceedings of the Tenth Conference on Computational Natural Language Learning (CoNLL-X)