John Aberdeen


2019

pdf bib
Scalable Methods for Annotating Legal-Decision Corpora
Lisa Ferro | John Aberdeen | Karl Branting | Craig Pfeifer | Alexander Yeh | Amartya Chakraborty
Proceedings of the Natural Legal Language Processing Workshop 2019

Recent research has demonstrated that judicial and administrative decisions can be predicted by machine-learning models trained on prior decisions. However, to have any practical application, these predictions must be explainable, which in turn requires modeling a rich set of features. Such approaches face a roadblock if the knowledge engineering required to create these features is not scalable. We present an approach to developing a feature-rich corpus of administrative rulings about domain name disputes, an approach which leverages a small amount of manual annotation and prototypical patterns present in the case documents to automatically extend feature labels to the entire corpus. To demonstrate the feasibility of this approach, we report results from systems trained on this dataset.

2010

pdf bib
Evaluation of Machine Translation Errors in English and Iraqi Arabic
Sherri Condon | Dan Parvaz | John Aberdeen | Christy Doran | Andrew Freeman | Marwan Awad
Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)

Errors in machine translations of English-Iraqi Arabic dialogues were analyzed at two different points in the systems? development using HTER methods to identify errors and human annotations to refine TER annotations. The analyses were performed on approximately 100 translations into each language from 4 translation systems collected at two annual evaluations. Although the frequencies of errors in the more mature systems were lower, the proportions of error types exhibited little change. Results include high frequencies of pronoun errors in translations to English, high frequencies of subject person inflection in translations to Iraqi Arabic, similar frequencies of word order errors in both translation directions, and very low frequencies of polarity errors. The problems with many errors can be generalized as the need to insert lexemes not present in the source or vice versa, which includes errors in multi-word expressions. Discourse context will be required to resolve some problems with deictic elements like pronouns.

pdf bib
Measuring Risk and Information Preservation: Toward New Metrics for De-identification of Clinical Texts
Lynette Hirschman | John Aberdeen
Proceedings of the NAACL HLT 2010 Second Louhi Workshop on Text and Data Mining of Health Documents

2009

pdf bib
Normalization for Automated Metrics: English and Arabic Speech Translation
Sherri Condon | Gregory A. Sanders | Dan Parvaz | Alan Rubenstein | Christy Doran | John Aberdeen | Beatrice Oshika
Proceedings of Machine Translation Summit XII: Papers

2008

pdf bib
Applying Automated Metrics to Speech Translation Dialogs
Sherri Condon | Jon Phillips | Christy Doran | John Aberdeen | Dan Parvaz | Beatrice Oshika | Greg Sanders | Craig Schlenoff
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)

Over the past five years, the Defense Advanced Research Projects Agency (DARPA) has funded development of speech translation systems for tactical applications. A key component of the research program has been extensive system evaluation, with dual objectives of assessing progress overall and comparing among systems. This paper describes the methods used to obtain BLEU, TER, and METEOR scores for two-way English-Iraqi Arabic systems. We compare the scores with measures based on human judgments and demonstrate the effects of normalization operations on BLEU scores. Issues that are highlighted include the quality of test data and differential results of applying automated metrics to Arabic vs. English.

2001

pdf bib
Finding Errors Automatically in Semantically Tagged Dialogues
John Aberdeen | Christine Doran | Laurie Damianos | Samuel Bayer | Lynette Hirschman
Proceedings of the First International Conference on Human Language Technology Research

pdf bib
Comparing Several Aspects of Human-Computer and Human-Human Dialogues
Christine Doran | John Aberdeen | Laurie Damianos | Lynette Hirschman
Proceedings of the Second SIGdial Workshop on Discourse and Dialogue

2000

pdf bib
Evaluation for Darpa Communicator Spoken Dialogue Systems
Marilyn Walker | Lynette Hirschman | John Aberdeen
Proceedings of the Second International Conference on Language Resources and Evaluation (LREC’00)

1997

pdf bib
Mixed-Initiative Development of Language Processing Systems
David Day | John Aberdeen | Lynette Hirschman | Robyn Kozierok | Patricia Robinson | Marc Vilain
Fifth Conference on Applied Natural Language Processing

1996

pdf bib
MITRE: Description of the Alembic System as Used in MET
John Aberdeen | John Burger | David Day | Lynette Hirschman | David Palmer | Patricia Robinson | Marc Vilain
TIPSTER TEXT PROGRAM PHASE II: Proceedings of a Workshop held at Vienna, Virginia, May 6-8, 1996

1995

pdf bib
A Model-Theoretic Coreference Scoring Scheme
Marc Vilain | John Burger | John Aberdeen | Dennis Connolly | Lynette Hirschman
Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995

pdf bib
MITRE: Description of the Alembic System Used for MUC-6
John Aberdeen | John Burger | David Day | Lynette Hirschman | Patricia Robinson | Marc Vilain
Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995

1993

pdf bib
MITRE-Bedford:Description of theAlembicSystem as Used for MUC-5
John Aberdeen | John Burger | Dennis Connolly | Susan Roberts | Marc Vilain
Fifth Message Understanding Conference (MUC-5): Proceedings of a Conference Held in Baltimore, Maryland, August 25-27, 1993

1992

pdf bib
MITRE-Bedford ALEMBIC: MUC-4 Test Results and Analysis
John Aberdeen | John Burger | Dennis Connolly | Susan Roberts | Marc Vilain
Fourth Message Understanding Conference (MUC-4): Proceedings of a Conference Held in McLean, Virginia, June 16-18, 1992

pdf bib
MITRE-Bedford: Description of the ALEMBIC System as Used for MUC-4
John Aberdeen | John Burger | Dennis Connolly | Susan Roberts | Marc Vilain
Fourth Message Understanding Conference (MUC-4): Proceedings of a Conference Held in McLean, Virginia, June 16-18, 1992