Alex Kulesza


2014

pdf bib
A Repository of State of the Art and Competitive Baseline Summaries for Generic News Summarization
Kai Hong | John Conroy | Benoit Favre | Alex Kulesza | Hui Lin | Ani Nenkova
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)

In the period since 2004, many novel sophisticated approaches for generic multi-document summarization have been developed. Intuitive simple approaches have also been shown to perform unexpectedly well for the task. Yet it is practically impossible to compare the existing approaches directly, because systems have been evaluated on different datasets, with different evaluation measures, against different sets of comparison systems. Here we present a corpus of summaries produced by several state-of-the-art extractive summarization systems or by popular baseline systems. The inputs come from the 2004 DUC evaluation, the latest year in which generic summarization was addressed in a shared task. We use the same settings for ROUGE automatic evaluation to compare the systems directly and analyze the statistical significance of the differences in performance. We show that in terms of average scores the state-of-the-art systems appear similar but that in fact they produce very different summaries. Our corpus will facilitate future research on generic summarization and motivates the need for development of more sensitive evaluation measures and for approaches to system combination in summarization.

2012

pdf bib
Discovering Diverse and Salient Threads in Document Collections
Jennifer Gillenwater | Alex Kulesza | Ben Taskar
Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning

2009

pdf bib
Multi-Class Confidence Weighted Algorithms
Koby Crammer | Mark Dredze | Alex Kulesza
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

2004

pdf bib
Confidence Estimation for Machine Translation
John Blatz | Erin Fitzgerald | George Foster | Simona Gandrabur | Cyril Goutte | Alex Kulesza | Alberto Sanchis | Nicola Ueffing
COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics

pdf bib
A learning approach to improving sentence-level MT evaluation
Alex Kulesza | Stuart M. Shieber
Proceedings of the 10th Conference on Theoretical and Methodological Issues in Machine Translation of Natural Languages