Richard Schwartz

Also published as: R. Schwartz, Rich Schwartz


2020

pdf bib
Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020)
Kathy McKeown | Douglas W. Oard | Elizabeth | Richard Schwartz
Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020)

pdf bib
Reformulating Information Retrieval from Speech and Text as a Detection Problem
Damianos Karakos | Rabih Zbib | William Hartmann | Richard Schwartz | John Makhoul
Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020)

In the IARPA MATERIAL program, information retrieval (IR) is treated as a hard detection problem; the system has to output a single global ranking over all queries, and apply a hard threshold on this global list to come up with all the hypothesized relevant documents. This means that how queries are ranked relative to each other can have a dramatic impact on performance. In this paper, we study such a performance measure, the Average Query Weighted Value (AQWV), which is a combination of miss and false alarm rates. AQWV requires that the same detection threshold is applied to all queries. Hence, detection scores of different queries should be comparable, and, to do that, a score normalization technique (commonly used in keyword spotting from speech) should be used. We describe unsupervised methods for score normalization, which are borrowed from the speech field and adapted accordingly for IR, and demonstrate that they greatly improve AQWV on the task of cross-language information retrieval (CLIR), on three low-resource languages used in MATERIAL. We also present a novel supervised score normalization approach which gives additional gains.

pdf bib
The 2019 BBN Cross-lingual Information Retrieval System
Le Zhang | Damianos Karakos | William Hartmann | Manaj Srivastava | Lee Tarlin | David Akodes | Sanjay Krishna Gouda | Numra Bathool | Lingjun Zhao | Zhuolin Jiang | Richard Schwartz | John Makhoul
Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020)

In this paper, we describe a cross-lingual information retrieval (CLIR) system that, given a query in English, and a set of audio and text documents in a foreign language, can return a scored list of relevant documents, and present findings in a summary form in English. Foreign audio documents are first transcribed by a state-of-the-art pretrained multilingual speech recognition model that is finetuned to the target language. For text documents, we use multiple multilingual neural machine translation (MT) models to achieve good translation results, especially for low/medium resource languages. The processed documents and queries are then scored using a probabilistic CLIR model that makes use of the probability of translation from GIZA translation tables and scores from a Neural Network Lexical Translation Model (NNLTM). Additionally, advanced score normalization, combination, and thresholding schemes are employed to maximize the Average Query Weighted Value (AQWV) scores. The CLIR output, together with multiple translation renderings, are selected and translated into English snippets via a summarization model. Our turnkey system is language agnostic and can be quickly trained for a new low-resource language in few days.

pdf bib
What Set of Documents to Present to an Analyst?
Richard Schwartz | John Makhoul | Lee Tarlin | Damianos Karakos
Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020)

We describe the human triage scenario envisioned in the Cross-Lingual Information Retrieval (CLIR) problem of the [REDUCT] Program. The overall goal is to maximize the quality of the set of documents that is given to a bilingual analyst, as measured by the AQWV score. The initial set of source documents that are retrieved by the CLIR system is summarized in English and presented to human judges who attempt to remove the irrelevant documents (false alarms); the resulting documents are then presented to the analyst. First, we describe the AQWV performance measure and show that, in our experience, if the acceptance threshold of the CLIR component has been optimized to maximize AQWV, the loss in AQWV due to false alarms is relatively constant across many conditions, which also limits the possible gain that can be achieved by any post filter (such as human judgments) that removes false alarms. Second, we analyze the likely benefits for the triage operation as a function of the initial CLIR AQWV score and the ability of the human judges to remove false alarms without removing relevant documents. Third, we demonstrate that we can increase the benefit for human judgments by combining the human judgment scores with the original document scores returned by the automatic CLIR system.

2015

pdf bib
Statistical Machine Translation Features with Multitask Tensor Networks
Hendra Setiawan | Zhongqiang Huang | Jacob Devlin | Thomas Lamar | Rabih Zbib | Richard Schwartz | John Makhoul
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)

2014

pdf bib
Morphological Segmentation for Keyword Spotting
Karthik Narasimhan | Damianos Karakos | Richard Schwartz | Stavros Tsakalidis | Regina Barzilay
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)

pdf bib
Fast and Robust Neural Network Joint Models for Statistical Machine Translation
Jacob Devlin | Rabih Zbib | Zhongqiang Huang | Thomas Lamar | Richard Schwartz | John Makhoul
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

2013

pdf bib
Systematic Comparison of Professional and Crowdsourced Reference Translations for Machine Translation
Rabih Zbib | Gretchen Markiewicz | Spyros Matsoukas | Richard Schwartz | John Makhoul
Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

2012

pdf bib
Machine Translation of Arabic Dialects
Rabih Zbib | Erika Malchiodi | Jacob Devlin | David Stallard | Spyros Matsoukas | Richard Schwartz | John Makhoul | Omar F. Zaidan | Chris Callison-Burch
Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

2011

pdf bib
Expected BLEU Training for Graphs: BBN System Description for WMT11 System Combination Task
Antti-Veikko Rosti | Bing Zhang | Spyros Matsoukas | Richard Schwartz
Proceedings of the Sixth Workshop on Statistical Machine Translation

pdf bib
Improving Low-Resource Statistical Machine Translation with a Novel Semantic Word Clustering Algorithm
Jeff Ma | Spyros Matsoukas | Richard Schwartz
Proceedings of Machine Translation Summit XIII: Papers

2010

pdf bib
BBN System Description for WMT10 System Combination Task
Antti-Veikko Rosti | Bing Zhang | Spyros Matsoukas | Richard Schwartz
Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR

pdf bib
Decision Trees for Lexical Smoothing in Statistical Machine Translation
Rabih Zbib | Spyros Matsoukas | Richard Schwartz | John Makhoul
Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR

2009

pdf bib
Incremental Hypothesis Alignment with Flexible Matching for Building Confusion Networks: BBN System Description for WMT09 System Combination Task
Antti-Veikko Rosti | Bing Zhang | Spyros Matsoukas | Richard Schwartz
Proceedings of the Fourth Workshop on Statistical Machine Translation

pdf bib
Fluency, Adequacy, or HTER? Exploring Different Human Judgments with a Tunable MT Metric
Matthew Snover | Nitin Madnani | Bonnie Dorr | Richard Schwartz
Proceedings of the Fourth Workshop on Statistical Machine Translation

2008

pdf bib
Incremental Hypothesis Alignment for Building Confusion Networks with Application to Machine Translation System Combination
Antti-Veikko Rosti | Bing Zhang | Spyros Matsoukas | Richard Schwartz
Proceedings of the Third Workshop on Statistical Machine Translation

pdf bib
Language and Translation Model Adaptation using Comparable Corpora
Matthew Snover | Bonnie Dorr | Richard Schwartz
Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing

pdf bib
Are Multiple Reference Translations Necessary? Investigating the Value of Paraphrased Reference Translations in Parameter Optimization
Nitin Madnani | Philip Resnik | Bonnie J. Dorr | Richard Schwartz
Proceedings of the 8th Conference of the Association for Machine Translation in the Americas: Research Papers

Most state-of-the-art statistical machine translation systems use log-linear models, which are defined in terms of hypothesis features and weights for those features. It is standard to tune the feature weights in order to maximize a translation quality metric, using held-out test sentences and their corresponding reference translations. However, obtaining reference translations is expensive. In our earlier work (Madnani et al., 2007), we introduced a new full-sentence paraphrase technique, based on English-to-English decoding with an MT system, and demonstrated that the resulting paraphrases can be used to cut the number of human reference translations needed in half. In this paper, we take the idea a step further, asking how far it is possible to get with just a single good reference translation for each item in the development set. Our analysis suggests that it is necessary to invest in four or more human translations in order to significantly improve on a single translation augmented by monolingual paraphrases.

2007

pdf bib
Combining Outputs from Multiple Machine Translation Systems
Antti-Veikko Rosti | Necip Fazil Ayan | Bing Xiang | Spyros Matsoukas | Richard Schwartz | Bonnie Dorr
Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference

pdf bib
Improved Word-Level System Combination for Machine Translation
Antti-Veikko Rosti | Spyros Matsoukas | Richard Schwartz
Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics

2006

pdf bib
A Study of Translation Edit Rate with Targeted Human Annotation
Matthew Snover | Bonnie Dorr | Rich Schwartz | Linnea Micciulla | John Makhoul
Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers

We examine a new, intuitive measure for evaluating machine-translation output that avoids the knowledge intensiveness of more meaning-based approaches, and the labor-intensiveness of human judgments. Translation Edit Rate (TER) measures the amount of editing that a human would have to perform to change a system output so it exactly matches a reference translation. We show that the single-reference variant of TER correlates as well with human judgments of MT quality as the four-reference variant of BLEU. We also define a human-targeted TER (or HTER) and show that it yields higher correlations with human judgments than BLEU—even when BLEU is given human-targeted references. Our results indicate that HTER correlates with human judgments better than HMETEOR and that the four-reference variants of TER and HTER correlate with human judgments as well as—or better than—a second human judgment does.

2005

pdf bib
A Methodology for Extrinsic Evaluation of Text Summarization: Does ROUGE Correlate?
Bonnie Dorr | Christof Monz | Stacy President | Richard Schwartz | David Zajic
Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization

2004

pdf bib
Using N-best lists for Named Entity Recognition from Chinese Speech
Lufeng Zhai | Pascale Fung | Richard Schwartz | Marine Carpuat | Dekai Wu
Proceedings of HLT-NAACL 2004: Short Papers

pdf bib
A Lexically-Driven Algorithm for Disfluency Detection
Matthew Snover | Bonnie Dorr | Richard Schwartz
Proceedings of HLT-NAACL 2004: Short Papers

2003

pdf bib
Hedge Trimmer: A Parse-and-Trim Approach to Headline Generation
Bonnie Dorr | David Zajic | Richard Schwartz
Proceedings of the HLT-NAACL 03 Text Summarization Workshop

2000

pdf bib
Annotating Resources for Information Extraction
Sean Boisen | Michael R. Crystal | Richard Schwartz | Rebecca Stone | Ralph Weischedel
Proceedings of the Second International Conference on Language Resources and Evaluation (LREC’00)

pdf bib
Named Entity Extraction from Noisy Input: Speech and OCR
David Miller | Sean Boisen | Richard Schwartz | Rebecca Stone | Ralph Weischedel
Sixth Applied Natural Language Processing Conference

1999

pdf bib
Why Doesn’t Natural Language Come Naturally?
Richard Schwartz
1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora

1998

pdf bib
BBN: Description of the SIFT System as Used for MUC-7
Scott Miller | Michael Crystal | Heidi Fox | Lance Ramshaw | Richard Schwartz | Rebecca Stone | Ralph Weischedel | The Annotation Group
Seventh Message Understanding Conference (MUC-7): Proceedings of a Conference Held in Fairfax, Virginia, April 29 - May 1, 1998

pdf bib
Algorithms That Learn to Extract Information BBN: TIPSTER Phase III
Scott Miller | Michael Crystal | Heidi Fox | Lance Ramshaw | Richard Schwartz | Rebecca Stone | Ralph Weischedel
TIPSTER TEXT PROGRAM PHASE III: Proceedings of a Workshop held at Baltimore, Maryland, October 13-15, 1998

1997

pdf bib
Nymble: a High-Performance Learning Name-finder
Daniel M. Bikel | Scott Miller | Richard Schwartz | Ralph Weischedel
Fifth Conference on Applied Natural Language Processing

1996

pdf bib
A Fully Statistical Approach to Natural Language Interfaces
Scott Miller | David Stallard | Robert Bobrow | Richard Schwartz
34th Annual Meeting of the Association for Computational Linguistics

1994

pdf bib
Hidden Understanding Models of Natural Language
Scott Miller | Robert Bobrow | Robert Ingria | Richard Schwartz
32nd Annual Meeting of the Association for Computational Linguistics

pdf bib
On Using Written Language Training Data for Spoken Language Modeling
R. Schwartz | L. Nguyen | F. Kubala | G. Chou | G. Zavaliagkos | J. Makhoul
Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994

pdf bib
Statistical Language Processing Using Hidden Understanding Models
Scott Miller | Richard Schwartz | Robert Bobrow | Robert Ingria
Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994

pdf bib
Adaptation to New Microphones Using Tied-Mixture Normalization
Anastasios Anastasakos | Francis Kubala | John Makhoul | Richard Schwartz
Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994

pdf bib
Session 13: CSR Search
Richard Schwartz
Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994

pdf bib
Is N-Best Dead?
Long Nguyen | Richard Schwartz | Ying Zhao | George Zavaliagkos
Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994

pdf bib
On-Line Cursive Handwriting Recognition Using Hidden Markov Models and Statistical Grammars
John Makhoul | Thad Starner | Richard Schwartz | George Chou
Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994

pdf bib
Robust Continuous Speech Recognition
John Makhoul | Richard Schwartz
Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994

1993

pdf bib
Coping with Ambiguity and Unknown Words through Probabilistic Models
Ralph Weischedel | Marie Meteer | Richard Schwartz | Lance Ramshaw | Jeff Palmucci
Computational Linguistics, Volume 19, Number 2, June 1993, Special Issue on Using Large Corpora: II

pdf bib
Comparative Experiments on Large Vocabulary Speech Recognition
Richard Schwartz | Tasos Anastasakos | Francis Kubala | John Makhoul | Long Nguyen | George Zavaliagkos
Human Language Technology: Proceedings of a Workshop Held at Plainsboro, New Jersey, March 21-24, 1993

pdf bib
Search Algorithms for Software-Only Real-Time Recognition with Very Large Vocabularies
Long Nguyen | Richard Schwartz | Francis Kubala | Paul Placeway
Human Language Technology: Proceedings of a Workshop Held at Plainsboro, New Jersey, March 21-24, 1993

pdf bib
Robust Continuous Speech Recognition
John Makhoul | Richard Schwartz
Human Language Technology: Proceedings of a Workshop Held at Plainsboro, New Jersey, March 21-24, 1993

1992

pdf bib
BBN BYBLOS and HARC February 1992 ATIS Benchmark Results
Francis Kubala | Chris Barry | Madeleine Bates | Robert Bobrow | Pascale Fung | Robert Ingria | John Makhoul | Long Nguyen | Richard Schwartz | David Stallard
Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23-26, 1992

pdf bib
Improving State-of-the-Art Continuous Speech Recognition Systems Using the N-Best Paradigm with Neural Networks
S. Austin | G. Zavaliagkos | J. Makhoul | R. Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23-26, 1992

pdf bib
BBN Real-Time Speech Recognition Demonstrations
Steve Austin | Rusty Bobrow | Dan Ellard | Robert Ingria | John Makhoul | Long Nguyen | Pat Peterson | Paul Placeway | Richard Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23-26, 1992

pdf bib
Robust Continuous Speech Recognition
John Makhoul | Richard Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23-26, 1992

1991

pdf bib
BYBLOS Speech Recognition Benchmark Results
F. Kubala | S. Austin | C. Barry | J. Makhoul | P. Placeway | R. Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Pacific Grove, California, February 19-22, 1991

pdf bib
Integration of Diverse Recognition Methodologies Through Reevaluation of N-Best Sentence Hypotheses
M. Ostendorf | A. Kannan | S. Austin | O. Kimball | R. Schwartz | J.R. Rohlicek
Speech and Natural Language: Proceedings of a Workshop Held at Pacific Grove, California, February 19-22, 1991

pdf bib
BBN HARC and DELPHI Results on the ATIS Benchmarks - February 1991
S. Austin | D. Ayuso | M. Bates | R. Bobrow | R. Ingria | J. Makhoul | P. Placeway | R. Schwartz | D. Stallard
Speech and Natural Language: Proceedings of a Workshop Held at Pacific Grove, California, February 19-22, 1991

pdf bib
Continuous Speech Recognition Using Segmental Neural Nets
S. Austin | J. Makhoul | R. Schwartz | G. Zavaliagkos
Speech and Natural Language: Proceedings of a Workshop Held at Pacific Grove, California, February 19-22, 1991

pdf bib
Studies in Part of Speech Labelling
Marie Meteer | Richard Schwartz | Ralph Weischedel
Speech and Natural Language: Proceedings of a Workshop Held at Pacific Grove, California, February 19-22, 1991

pdf bib
Research in Continuous Speech Recognition
John Makhoul | Richard Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Pacific Grove, California, February 19-22, 1991

1990

pdf bib
Efficient, High-Performance Algorithms for N-Best Search
Richard Schwartz | Steve Austin
Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990

pdf bib
Toward a Real-Time Spoken Language System Using Commercial Hardware
Steve Austin | Pat Peterson | Paul Placeway | Richard Schwartz | Jeff Vandergrift
Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990

pdf bib
A New Paradigm for Speaker-Independent Training and Speaker Adaptation
Francis Kubala | Richard Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990

pdf bib
Towards Understanding Text with a Very Large Vocabulary
Damaris Ayuso | R. Bobrow | Dawn MacLaughlin | Marie Meteer | Lance Ramshaw | Rich Schwartz | Ralph Weischedel
Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990

pdf bib
Research in Continuous Speech Recognition
John Makhoul | Richard Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990

1989

pdf bib
Research in Continuous Speech Recognition
John Makhoul | Richard Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Philadelphia, Pennsylvania, February 21-23, 1989

pdf bib
The BBN BYBLOS Continuous Speech Recognition System
Richard Schwartz | Chris Barry | Yen-Lu Chow | Alan Deft | Ming-Whei Feng | Owen Kimball | Francis Kubala | John Makhoul | Jeffrey Vandegrift
Speech and Natural Language: Proceedings of a Workshop Held at Philadelphia, Pennsylvania, February 21-23, 1989

pdf bib
Speaker Adaptation from Limited Training in the BBN BYBLOS Speech Recognition System
Francis Kubala | Ming-Whei Feng | John Makhoul | Richard Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Philadelphia, Pennsylvania, February 21-23, 1989

pdf bib
A Simple Statistical Class Grammar for Measuring Speech Recognition Performance
Alan Derr | Richard Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Cape Cod, Massachusetts, October 15-18, 1989

pdf bib
The N-Best Algorithm: Efficient Procedure for Finding Top N Sentence Hypotheses
Yen-Lu Chow | Richard Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Cape Cod, Massachusetts, October 15-18, 1989

pdf bib
Improved HMM Models for High Performance Speech Recognition
Steve Austin | Chris Barry | Yen-Lu Chow | Man Derr | Owen Kimball | Francis Kubala | John Makhoul | Paul Placeway | William Russell | Richard Schwartz | George Yu
Speech and Natural Language: Proceedings of a Workshop Held at Cape Cod, Massachusetts, October 15-18, 1989

pdf bib
Speaker Adaptation Using Multiple Reference Speakers
Francis Kubala | Richard Schwartz | Chris Barry
Speech and Natural Language: Proceedings of a Workshop Held at Cape Cod, Massachusetts, October 15-18, 1989

pdf bib
Automatic Detection Of New Words In A Large Vocabulary Continuous Speech Recognition System
Ayman Asadi | Richard Schwartz | John Makhoul
Speech and Natural Language: Proceedings of a Workshop Held at Cape Cod, Massachusetts, October 15-18, 1989

pdf bib
Summary of Session on Hardware for Spoken Language Demonstrations
Richard Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Cape Cod, Massachusetts, October 15-18, 1989

pdf bib
Research in Continuous Speech Recognition
John Makhoul | Richard Schwartz
Speech and Natural Language: Proceedings of a Workshop Held at Cape Cod, Massachusetts, October 15-18, 1989