Teruko Mitamura


2021

pdf bib
A Survey of Data Augmentation Approaches for NLP
Steven Y. Feng | Varun Gangal | Jason Wei | Sarath Chandar | Soroush Vosoughi | Teruko Mitamura | Eduard Hovy
Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

2020

pdf bib
Extraction of the Argument Structure of Tokyo Metropolitan Assembly Minutes: Segmentation of Question-and-Answer Sets
Keiichi Takamaru | Yasutomo Kimura | Hideyuki Shibuki | Hokuto Ototake | Yuzu Uchida | Kotaro Sakamoto | Madoka Ishioroshi | Teruko Mitamura | Noriko Kando
Proceedings of the 12th Language Resources and Evaluation Conference

In this study, we construct a corpus of Japanese local assembly minutes. All speeches in an assembly were transcribed into a local assembly minutes based on the local autonomy law. Therefore, the local assembly minutes form an extremely large amount of text data. Our ultimate objectives were to summarize and present the arguments in the assemblies, and to use the minutes as primary information for arguments in local politics. To achieve this, we structured all statements in assembly minutes. We focused on the structure of the discussion, i.e., the extraction of question and answer pairs. We organized the shared task “QA Lab-PoliInfo” in NTCIR 14. We conducted a “segmentation task” to identify the scope of one question and answer in the minutes as a sub task of the shared task. For the segmentation task, 24 runs from five teams were submitted. Based on the obtained results, the best recall was 1.000, best precision was 0.940, and best F-measure was 0.895.

pdf bib
Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events
Claire Bonial | Tommaso Caselli | Snigdha Chaturvedi | Elizabeth Clark | Ruihong Huang | Mohit Iyyer | Alejandro Jaimes | Heng Ji | Lara J. Martin | Ben Miller | Teruko Mitamura | Nanyun Peng | Joel Tetreault
Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events

pdf bib
A Data-Centric Framework for Composable NLP Workflows
Zhengzhong Liu | Guanxiong Ding | Avinash Bukkittu | Mansi Gupta | Pengzhi Gao | Atif Ahmed | Shikun Zhang | Xin Gao | Swapnil Singhavi | Linwei Li | Wei Wei | Zecong Hu | Haoran Shi | Xiaodan Liang | Teruko Mitamura | Eric Xing | Zhiting Hu
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

Empirical natural language processing (NLP) systems in application domains (e.g., healthcare, finance, education) involve interoperation among multiple components, ranging from data ingestion, human annotation, to text retrieval, analysis, generation, and visualization. We establish a unified open-source framework to support fast development of such sophisticated NLP workflows in a composable manner. The framework introduces a uniform data representation to encode heterogeneous results by a wide range of NLP tasks. It offers a large repository of processors for NLP tasks, visualization, and annotation, which can be easily assembled with full interoperability under the unified representation. The highly extensible framework allows plugging in custom processors from external off-the-shelf NLP and deep learning libraries. The whole framework is delivered through two modularized yet integratable open-source projects, namely Forte (for workflow infrastructure and NLP function processors) and Stave (for user interaction, visualization, and annotation).

pdf bib
GenAug: Data Augmentation for Finetuning Text Generators
Steven Y. Feng | Varun Gangal | Dongyeop Kang | Teruko Mitamura | Eduard Hovy
Proceedings of Deep Learning Inside Out (DeeLIO): The First Workshop on Knowledge Extraction and Integration for Deep Learning Architectures

In this paper, we investigate data augmentation for text generation, which we call GenAug. Text generation and language modeling are important tasks within natural language processing, and are especially challenging for low-data regimes. We propose and evaluate various augmentation methods, including some that incorporate external knowledge, for finetuning GPT-2 on a subset of Yelp Reviews. We also examine the relationship between the amount of augmentation and the quality of the generated text. We utilize several metrics that evaluate important aspects of the generated text including its diversity and fluency. Our experiments demonstrate that insertion of character-level synthetic noise and keyword replacement with hypernyms are effective augmentation methods, and that the quality of generations improves to a peak at approximately three times the amount of original data.

2019

pdf bib
Pentagon at MEDIQA 2019: Multi-task Learning for Filtering and Re-ranking Answers using Language Inference and Question Entailment
Hemant Pugaliya | Karan Saxena | Shefali Garg | Sheetal Shalini | Prashant Gupta | Eric Nyberg | Teruko Mitamura
Proceedings of the 18th BioNLP Workshop and Shared Task

Parallel deep learning architectures like fine-tuned BERT and MT-DNN, have quickly become the state of the art, bypassing previous deep and shallow learning methods by a large margin. More recently, pre-trained models from large related datasets have been able to perform well on many downstream tasks by just fine-tuning on domain-specific datasets (similar to transfer learning). However, using powerful models on non-trivial tasks, such as ranking and large document classification, still remains a challenge due to input size limitations of parallel architecture and extremely small datasets (insufficient for fine-tuning). In this work, we introduce an end-to-end system, trained in a multi-task setting, to filter and re-rank answers in the medical domain. We use task-specific pre-trained models as deep feature extractors. Our model achieves the highest Spearman’s Rho and Mean Reciprocal Rank of 0.338 and 0.9622 respectively, on the ACL-BioNLP workshop MediQA Question Answering shared-task.

pdf bib
Dr.Quad at MEDIQA 2019: Towards Textual Inference and Question Entailment using contextualized representations
Vinayshekhar Bannihatti Kumar | Ashwin Srinivasan | Aditi Chaudhary | James Route | Teruko Mitamura | Eric Nyberg
Proceedings of the 18th BioNLP Workshop and Shared Task

This paper presents the submissions by TeamDr.Quad to the ACL-BioNLP 2019 shared task on Textual Inference and Question Entailment in the Medical Domain. Our system is based on the prior work Liu et al. (2019) which uses a multi-task objective function for textual entailment. In this work, we explore different strategies for generalizing state-of-the-art language understanding models to the specialized medical domain. Our results on the shared task demonstrate that incorporating domain knowledge through data augmentation is a powerful strategy for addressing challenges posed specialized domains such as medicine.

pdf bib
Sieg at MEDIQA 2019: Multi-task Neural Ensemble for Biomedical Inference and Entailment
Sai Abishek Bhaskar | Rashi Rungta | James Route | Eric Nyberg | Teruko Mitamura
Proceedings of the 18th BioNLP Workshop and Shared Task

This paper presents a multi-task learning approach to natural language inference (NLI) and question entailment (RQE) in the biomedical domain. Recognizing textual inference relations and question similarity can address the issue of answering new consumer health questions by mapping them to Frequently Asked Questions on reputed websites like the NIH. We show that leveraging information from parallel tasks across domains along with medical knowledge integration allows our model to learn better biomedical feature representations. Our final models for the NLI and RQE tasks achieve the 4th and 2nd rank on the shared-task leaderboard respectively.

2018

pdf bib
Low-resource Cross-lingual Event Type Detection via Distant Supervision with Minimal Effort
Aldrian Obaja Muis | Naoki Otani | Nidhi Vyas | Ruochen Xu | Yiming Yang | Teruko Mitamura | Eduard Hovy
Proceedings of the 27th International Conference on Computational Linguistics

The use of machine learning for NLP generally requires resources for training. Tasks performed in a low-resource language usually rely on labeled data in another, typically resource-rich, language. However, there might not be enough labeled data even in a resource-rich language such as English. In such cases, one approach is to use a hand-crafted approach that utilizes only a small bilingual dictionary with minimal manual verification to create distantly supervised data. Another is to explore typical machine learning techniques, for example adversarial training of bilingual word representations. We find that in event-type detection task—the task to classify [parts of] documents into a fixed set of labels—they give about the same performance. We explore ways in which the two methods can be complementary and also see how to best utilize a limited budget for manual annotation to maximize performance gain.

pdf bib
Open-Domain Event Detection using Distant Supervision
Jun Araki | Teruko Mitamura
Proceedings of the 27th International Conference on Computational Linguistics

This paper introduces open-domain event detection, a new event detection paradigm to address issues of prior work on restricted domains and event annotation. The goal is to detect all kinds of events regardless of domains. Given the absence of training data, we propose a distant supervision method that is able to generate high-quality training data. Using a manually annotated event corpus as gold standard, our experiments show that despite no direct supervision, the model outperforms supervised models. This result indicates that the distant supervision enables robust event detection in various domains, while obviating the need for human annotation of events.

pdf bib
Graph Based Decoding for Event Sequencing and Coreference Resolution
Zhengzhong Liu | Teruko Mitamura | Eduard Hovy
Proceedings of the 27th International Conference on Computational Linguistics

Events in text documents are interrelated in complex ways. In this paper, we study two types of relation: Event Coreference and Event Sequencing. We show that the popular tree-like decoding structure for automated Event Coreference is not suitable for Event Sequencing. To this end, we propose a graph-based decoding algorithm that is applicable to both tasks. The new decoding algorithm supports flexible feature sets for both tasks. Empirically, our event coreference system has achieved state-of-the-art performance on the TAC-KBP 2015 event coreference task and our event sequencing system beats a strong temporal-based, oracle-informed baseline. We discuss the challenges of studying these event relations.

pdf bib
Parser combinators for Tigrinya and Oromo morphology
Patrick Littell | Tom McCoy | Na-Rae Han | Shruti Rijhwani | Zaid Sheikh | David Mortensen | Teruko Mitamura | Lori Levin
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)

pdf bib
BioAMA: Towards an End to End BioMedical Question Answering System
Vasu Sharma | Nitish Kulkarni | Srividya Pranavi | Gabriel Bayomi | Eric Nyberg | Teruko Mitamura
Proceedings of the BioNLP 2018 workshop

In this paper, we present a novel Biomedical Question Answering system, BioAMA: “Biomedical Ask Me Anything” on task 5b of the annual BioASQ challenge. In this work, we focus on a wide variety of question types including factoid, list based, summary and yes/no type questions that generate both exact and well-formed ‘ideal’ answers. For summary-type questions, we combine effective IR-based techniques for retrieval and diversification of relevant snippets for a question to create an end-to-end system which achieves a ROUGE-2 score of 0.72 and a ROUGE-SU4 score of 0.71 on ideal answer questions (7% improvement over the previous best model). Additionally, we propose a novel NLI-based framework to answer the yes/no questions. To train the NLI model, we also devise a transfer-learning technique by cross-domain projection of word embeddings. Finally, we present a two-stage approach to address the factoid and list type questions by first generating a candidate set using NER taggers and ranking them using both supervised or unsupervised techniques.

pdf bib
Proceedings of the Workshop Events and Stories in the News 2018
Tommaso Caselli | Ben Miller | Marieke van Erp | Piek Vossen | Martha Palmer | Eduard Hovy | Teruko Mitamura | David Caswell | Susan W. Brown | Claire Bonial
Proceedings of the Workshop Events and Stories in the News 2018

pdf bib
Interoperable Annotation of Events and Event Relations across Domains
Jun Araki | Lamana Mulaffer | Arun Pandian | Yukari Yamakawa | Kemal Oflazer | Teruko Mitamura
Proceedings 14th Joint ACL - ISO Workshop on Interoperable Semantic Annotation

pdf bib
Ontology-Based Retrieval & Neural Approaches for BioASQ Ideal Answer Generation
Ashwin Naresh Kumar | Harini Kesavamoorthy | Madhura Das | Pramati Kalwad | Khyathi Chandu | Teruko Mitamura | Eric Nyberg
Proceedings of the 6th BioASQ Workshop A challenge on large-scale biomedical semantic indexing and question answering

The ever-increasing magnitude of biomedical information sources makes it difficult and time-consuming for a human researcher to find the most relevant documents and pinpointed answers for a specific question or topic when using only a traditional search engine. Biomedical Question Answering systems automatically identify the most relevant documents and pinpointed answers, given an information need expressed as a natural language question. Generating a non-redundant, human-readable summary that satisfies the information need of a given biomedical question is the focus of the Ideal Answer Generation task, part of the BioASQ challenge. This paper presents a system for ideal answer generation (using ontology-based retrieval and a neural learning-to-rank approach, combined with extractive and abstractive summarization techniques) which achieved the highest ROUGE score of 0.659 on the BioASQ 5b batch 2 test.

pdf bib
Textual Entailment based Question Generation
Takaaki Matsumoto | Kimihiro Hasegawa | Yukari Yamakawa | Teruko Mitamura
Proceedings of the Workshop on Intelligent Interactive Systems and Language Generation (2IS&NLG)

pdf bib
Automatic Event Salience Identification
Zhengzhong Liu | Chenyan Xiong | Teruko Mitamura | Eduard Hovy
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

Identifying the salience (i.e. importance) of discourse units is an important task in language understanding. While events play important roles in text documents, little research exists on analyzing their saliency status. This paper empirically studies Event Salience and proposes two salience detection models based on discourse relations. The first is a feature based salience model that incorporates cohesion among discourse units. The second is a neural model that captures more complex interactions between discourse units. In our new large-scale event salience corpus, both methods significantly outperform the strong frequency baseline, while our neural model further improves the feature based one by a large margin. Our analyses demonstrate that our neural model captures interesting connections between salience and discourse unit relations (e.g., scripts and frame structures).

2017

pdf bib
Proceedings of the Events and Stories in the News Workshop
Tommaso Caselli | Ben Miller | Marieke van Erp | Piek Vossen | Martha Palmer | Eduard Hovy | Teruko Mitamura | David Caswell
Proceedings of the Events and Stories in the News Workshop

pdf bib
Event Detection Using Frame-Semantic Parser
Evangelia Spiliopoulou | Eduard Hovy | Teruko Mitamura
Proceedings of the Events and Stories in the News Workshop

Recent methods for Event Detection focus on Deep Learning for automatic feature generation and feature ranking. However, most of those approaches fail to exploit rich semantic information, which results in relatively poor recall. This paper is a small & focused contribution, where we introduce an Event Detection and classification system, based on deep semantic information retrieved from a frame-semantic parser. Our experiments show that our system achieves higher recall than state-of-the-art systems. Further, we claim that enhancing our system with deep learning techniques like feature ranking can achieve even better results, as it can benefit from both approaches.

2016

pdf bib
Proceedings of the Fourth Workshop on Events
Martha Palmer | Ed Hovy | Teruko Mitamura | Tim O’Gorman
Proceedings of the Fourth Workshop on Events

pdf bib
A Comparison of Event Representations in DEFT
Ann Bies | Zhiyi Song | Jeremy Getman | Joe Ellis | Justin Mott | Stephanie Strassel | Martha Palmer | Teruko Mitamura | Marjorie Freedman | Heng Ji | Tim O’Gorman
Proceedings of the Fourth Workshop on Events

pdf bib
Event Nugget and Event Coreference Annotation
Zhiyi Song | Ann Bies | Stephanie Strassel | Joe Ellis | Teruko Mitamura | Hoa Trang Dang | Yukari Yamakawa | Sue Holm
Proceedings of the Fourth Workshop on Events

pdf bib
Unsupervised Event Coreference for Abstract Words
Dheeraj Rajagopal | Eduard Hovy | Teruko Mitamura
Proceedings of the Workshop on Uphill Battles in Language Processing: Scaling Early Achievements to Robust Methods

pdf bib
Generating Questions and Multiple-Choice Answers using Semantic Analysis of Texts
Jun Araki | Dheeraj Rajagopal | Sreecharan Sankaranarayanan | Susan Holm | Yukari Yamakawa | Teruko Mitamura
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers

We present a novel approach to automated question generation that improves upon prior work both from a technology perspective and from an assessment perspective. Our system is aimed at engaging language learners by generating multiple-choice questions which utilize specific inference steps over multiple sentences, namely coreference resolution and paraphrase detection. The system also generates correct answers and semantically-motivated phrase-level distractors as answer choices. Evaluation by human annotators indicates that our approach requires a larger number of inference steps, which necessitate deeper semantic understanding of texts than a traditional single-sentence approach.

2015

pdf bib
Joint Event Trigger Identification and Event Coreference Resolution with Structured Perceptron
Jun Araki | Teruko Mitamura
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

pdf bib
Proceedings of the The 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation
Eduard Hovy | Teruko Mitamura | Martha Palmer
Proceedings of the The 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation

pdf bib
Evaluation Algorithms for Event Nugget Detection : A Pilot Study
Zhengzhong Liu | Teruko Mitamura | Eduard Hovy
Proceedings of the The 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation

pdf bib
Event Nugget Annotation: Processes and Issues
Teruko Mitamura | Yukari Yamakawa | Susan Holm | Zhiyi Song | Ann Bies | Seth Kulick | Stephanie Strassel
Proceedings of the The 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation

2014

pdf bib
Proceedings of the Second Workshop on EVENTS: Definition, Detection, Coreference, and Representation
Teruko Mitamura | Eduard Hovy | Martha Palmer
Proceedings of the Second Workshop on EVENTS: Definition, Detection, Coreference, and Representation

pdf bib
Evaluation for Partial Event Coreference
Jun Araki | Eduard Hovy | Teruko Mitamura
Proceedings of the Second Workshop on EVENTS: Definition, Detection, Coreference, and Representation

pdf bib
Resources for the Detection of Conventionalized Metaphors in Four Languages
Lori Levin | Teruko Mitamura | Brian MacWhinney | Davida Fromm | Jaime Carbonell | Weston Feely | Robert Frederking | Anatole Gershman | Carlos Ramirez
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)

This paper describes a suite of tools for extracting conventionalized metaphors in English, Spanish, Farsi, and Russian. The method depends on three significant resources for each language: a corpus of conventionalized metaphors, a table of conventionalized conceptual metaphors (CCM table), and a set of extraction rules. Conventionalized metaphors are things like “escape from poverty” and “burden of taxation”. For each metaphor, the CCM table contains the metaphorical source domain word (such as “escape”) the target domain word (such as “poverty”) and the grammatical construction in which they can be found. The extraction rules operate on the output of a dependency parser and identify the grammatical configurations (such as a verb with a prepositional phrase complement) that are likely to contain conventional metaphors. We present results on detection rates for conventional metaphors and analysis of the similarity and differences of source domains for conventional metaphors in the four languages.

pdf bib
Supervised Within-Document Event Coreference using Information Propagation
Zhengzhong Liu | Jun Araki | Eduard Hovy | Teruko Mitamura
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)

Event coreference is an important task for full text analysis. However, previous work uses a variety of approaches, sources and evaluation, making the literature confusing and the results incommensurate. We provide a description of the differences to facilitate future research. Second, we present a supervised method for event coreference resolution that uses a rich feature set and propagates information alternatively between events and their arguments, adapting appropriately for each type of argument.

pdf bib
Detecting Subevent Structure for Event Coreference Resolution
Jun Araki | Zhengzhong Liu | Eduard Hovy | Teruko Mitamura
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)

In the task of event coreference resolution, recent work has shown the need to perform not only full coreference but also partial coreference of events. We show that subevents can form a particular hierarchical event structure. This paper examines a novel two-stage approach to finding and improving subevent structures. First, we introduce a multiclass logistic regression model that can detect subevent relations in addition to full coreference. Second, we propose a method to improve subevent structure based on subevent clusters detected by the model. Using a corpus in the Intelligence Community domain, we show that the method achieves over 3.2 BLANC F1 gain in detecting subevent relations against the logistic regression model.

2013

pdf bib
An English Reading Tool as a NLP Showcase
Mahmoud Azab | Ahmed Salama | Kemal Oflazer | Hideki Shima | Jun Araki | Teruko Mitamura
The Companion Volume of the Proceedings of IJCNLP 2013: System Demonstrations

pdf bib
Workshop on Events: Definition, Detection, Coreference, and Representation
Eduard Hovy | Teruko Mitamura | Martha Palmer
Workshop on Events: Definition, Detection, Coreference, and Representation

pdf bib
Events are Not Simple: Identity, Non-Identity, and Quasi-Identity
Eduard Hovy | Teruko Mitamura | Felisa Verdejo | Jun Araki | Andrew Philpot
Workshop on Events: Definition, Detection, Coreference, and Representation

pdf bib
An NLP-based Reading Tool for Aiding Non-native English Readers
Mahmoud Azab | Ahmed Salama | Kemal Oflazer | Hideki Shima | Jun Araki | Teruko Mitamura
Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013

2012

pdf bib
Diversifiable Bootstrapping for Acquiring High-Coverage Paraphrase Resource
Hideki Shima | Teruko Mitamura
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)

Recognizing similar or close meaning on different surface form is a common challenge in various Natural Language Processing and Information Access applications. However, we identified multiple limitations in existing resources that can be used for solving the vocabulary mismatch problem. To this end, we will propose the Diversifiable Bootstrapping algorithm that can learn paraphrase patterns with a high lexical coverage. The algorithm works in a lightly-supervised iterative fashion, where instance and pattern acquisition are interleaved, each using information provided by the other. By tweaking a parameter in the algorithm, resulting patterns can be diversifiable with a specific degree one can control.

2011

pdf bib
Diversity-aware Evaluation for Paraphrase Patterns
Hideki Shima | Teruko Mitamura
Proceedings of the TextInfer 2011 Workshop on Textual Entailment

2010

pdf bib
Automatic Collocation Suggestion in Academic Writing
Jian-Cheng Wu | Yu-Chia Chang | Teruko Mitamura | Jason S. Chang
Proceedings of the ACL 2010 Conference Short Papers

2009

pdf bib
Committed Belief Annotation and Tagging
Mona Diab | Lori Levin | Teruko Mitamura | Owen Rambow | Vinodkumar Prabhakaran | Weiwei Guo
Proceedings of the Third Linguistic Annotation Workshop (LAW III)

2007

pdf bib
What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA
Mengqiu Wang | Noah A. Smith | Teruko Mitamura
Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL)

pdf bib
Language-independent Probabilistic Answer Ranking for Question Answering
Jeongwoo Ko | Teruko Mitamura | Eric Nyberg
Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics

2006

pdf bib
A Fast, Accurate Deterministic Parser for Chinese
Mengqiu Wang | Kenji Sagae | Teruko Mitamura
Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics

pdf bib
Keyword Translation Accuracy and Cross-Lingual Question Answering inChinese and Japanese
Teruko Mitamura | Mengqiu Wang | Hideki Shima | Frank Lin
Proceedings of the Workshop on Multilingual Question Answering - MLQA ‘06

pdf bib
Parallel Syntactic Annotation of Multiple Languages
Owen Rambow | Bonnie Dorr | David Farwell | Rebecca Green | Nizar Habash | Stephen Helmreich | Eduard Hovy | Lori Levin | Keith J. Miller | Teruko Mitamura | Florence Reeder | Advaith Siddharthan
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)

This paper describes an effort to investigate the incrementally deepening development of an interlingua notation, validated by human annotation of texts in English plus six languages. We begin with deep syntactic annotation, and in this paper present a series of annotation manuals for six different languages at the deep-syntactic level of representation. Many syntactic differences between languages are removed in the proposed syntactic annotation, making them useful resources for multilingual NLP projects with semantic components.

pdf bib
Modular Approach to Error Analysis and Evaluation for Multilingual Question Answering
Hideki Shima | Mengqiu Wang | Frank Lin | Teruko Mitamura
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)

Multilingual Question Answering systems are generally very complex, integrating several sub-modules to achieve their result. Global metrics (such as average precision and recall) are insufficient when evaluating the performance of individual sub-modules and their influence on each other. In this paper, we present a modular approach to error analysis and evaluation; we use manually-constructed, gold-standard input for each module to obtain an upper-bound for the (local) performance of that module. This approach enables us to identify existing problem areas quickly, and to target improvements accordingly.

pdf bib
Analyzing the Effects of Spoken Dialog Systems on Driving Behavior
Jeongwoo Ko | Fumihiko Murase | Teruko Mitamura | Eric Nyberg | Masahiko Tateishi | Ichiro Akahori
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)

This paper presents an evaluation of a spoken dialog system for automotive environments. Our overall goal was to measure the impact of user-system interaction on the user’s driving performance, and to determine whether adding context-awareness to the dialog system might reduce the degree of user distraction during driving. To address this issue, we incorporated context-awareness into a spoken dialog system, and implemented three system features using user context, network context and dialog context. A series of experiments were conducted under three different configurations: driving without a dialog system, driving while using a context-aware dialog system, and driving while using a context-unaware dialog system. We measured the differences between the three configurations by comparing the average car speed, the frequency of speed changes and the angle between the car’s direction and the centerline on the road. These results indicate that context-awareness could reduce the degree of user distraction when using a dialog system during driving.

2004

pdf bib
Interlingual Annotation of Multilingual Text Corpora
Stephen Helmreich | David Farwell | Bonnie Dorr | Nizar Habash | Lori Levin | Teruko Mitamura | Florence Reeder | Keith Miller | Eduard Hovy | Owen Rambow | Advaith Siddharthan
Proceedings of the Workshop Frontiers in Corpus Annotation at HLT-NAACL 2004

pdf bib
Correction Grammars for Error Handling in a Speech Dialog System
Hirohiko Sagawa | Teruko Mitamura | Eric Nyberg
Proceedings of HLT-NAACL 2004: Short Papers

pdf bib
Keyword translation from English to Chinese for multilingual QA
Frank Lin | Teruko Mitamura
Proceedings of the 6th Conference of the Association for Machine Translation in the Americas: Technical Papers

The Keyword Translator is a part of the Question Analyzer module in the JAVELIN Question-Answering system; it translates the keywords, which are used to query documents and extract answers, from one language to another. Much work has been in the area of query translation for CLIR or MLIR, however, many have focused on methods using hard-to-obtain and domain-specific resources, and evaluation is often based on retrieval performance rather than translation correctness. In this paper we will describe methods combining easily accessible, general-purpose MT systems to improve keyword translation correctness. We also describe methods that utilize the question sentence available to a question-answering system to improve translation correctness. We will show that using multiple MT systems and the question sentence to translate keywords from English to Mandarin Chinese can significantly improve keyword translation correctness.

pdf bib
Interlingual annotation for MT development
Florence Reeder | Bonnie Dorr | David Farwell | Nizar Habash | Stephen Helmreich | Eduard Hovy | Lori Levin | Teruko Mitamura | Keith Miller | Owen Rambow | Advaith Siddharthan
Proceedings of the 6th Conference of the Association for Machine Translation in the Americas: Technical Papers

MT systems that use only superficial representations, including the current generation of statistical MT systems, have been successful and useful. However, they will experience a plateau in quality, much like other “silver bullet” approaches to MT. We pursue work on the development of interlingual representations for use in symbolic or hybrid MT systems. In this paper, we describe the creation of an interlingua and the development of a corpus of semantically annotated text, to be validated in six languages and evaluated in several ways. We have established a distributed, well-functioning research methodology, designed a preliminary interlingua notation, created annotation manuals and tools, developed a test collection in six languages with associated English translations, annotated some 150 translations, and designed and applied various annotation metrics. We describe the data sets being annotated and the interlingual (IL) representation language which uses two ontologies and a systematic theta-role list. We present the annotation tools built and outline the annotation process. Following this, we describe our evaluation methodology and conclude with a summary of issues that have arisen.

pdf bib
Pronominal Anaphora Resolution for Unrestricted Text
Anna Kupść | Teruko Mitamura | Benjamin Van Durme | Eric Nyberg
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)

pdf bib
An Information Repository Model for Advanced Question Answering Systems
Vasco Calais Pedro | Jeongwoo Ko | Eric Nyberg | Teruko Mitamura
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)

2003

pdf bib
Source language diagnostics for MT
Teruko Mitamura | Kathryn Baker | David Svoboda | Eric Nyberg
Proceedings of Machine Translation Summit IX: Papers

This paper presents a source language diagnostic system for controlled translation. Diagnostics were designed and implemented to address the most difficult rewrites for authors, based on an empirical analysis of log files containing over 180,000 sentences. The design and implementation of the diagnostic system are presented, along with experimental results from an empirical evaluation of the completed system. We found that the diagnostic system can correctly identify the problem in 90.2% of the cases. In addition, depending on the type of grammar problem, the diagnostic system may offer a rewritten sentence. We found that 89.4% of the rewritten sentences were correctly rewritten. The results suggest that these methods could be used as the basis for an automatic rewriting system in the future.

pdf bib
An integrated system for source language checking, analysis and term management
Eric Nyberg | Teruko Mitamura | David Svoboda | Jeongwoo Ko | Kathryn Baker | Jeffrey Micher
Proceedings of Machine Translation Summit IX: System Presentations

This paper presents an overview of the tools provided by KANTOO MT system for controlled source language checking, source text analysis, and terminology management. The steps in each process are described, and screen images are provided to illustrate the system architecture and example tool interfaces.

pdf bib
Teaching machine translation in a graduate language technologies program
Teruko Mitamura | Eric Nyberg | Robert Frederking
Workshop on Teaching Translation Technologies and Tools

This paper describes a graduate-level machine translation (MT) course taught at the Language Technologies Institute at Carnegie Mellon University. Most of the students in the course have a background in computer science. We discuss what we teach (the course syllabus), and how we teach it (lectures, homeworks, and projects). The course has evolved steadily over the past several years to incorporate refinements in the set of course topics, how they are taught, and how students “learn by doing”. The course syllabus has also evolved in response to changes in the field of MT and the role that MT plays in various social contexts.

pdf bib
Diagnostics for interactive controlled language checking
Teruko Mitamura | Kathryn Baker | Eric Nyberg | David Svoboda
EAMT Workshop: Improving MT through other language technology tools: resources and tools for building MT

2002

pdf bib
Design and Evolution of a Language Technologies Curriculum
Robert Frederking | Eric H. Nyberg | Teruko Mitamura | Jaime G. Carbonell
Proceedings of the ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics

pdf bib
Deriving semantic knowledge from descriptive texts using an MT system
Eric Nyberg | Teruko Mitamura | Kathryn Baker | David Svoboda | Brian Peterson | Jennifer Williams
Proceedings of the 5th Conference of the Association for Machine Translation in the Americas: Technical Papers

This paper describes the results of a feasibility study which focused on deriving semantic networks from descriptive texts using controlled language. The KANT system [3,6] was used to analyze input paragraphs, producing sentence-level interlingua representations. The interlinguas were merged to construct a paragraph-level representation, which was used to create a semantic network in Conceptual Graph (CG) [1] format. The interlinguas are also translated (using the KANTOO generator) into OWL statements for entry into the Ontology Works electrical power factbase [9]. The system was extended to allow simple querying in natural language.

pdf bib
Pronominal anaphora resolution in the KANTOO multilingual machine translation system
Teruko Mitamura | Eric Nyberg | Enrique Torrejon | Dave Svoboda | Annelen Brunner | Kathryn Baker
Proceedings of the 9th Conference on Theoretical and Methodological Issues in Machine Translation of Natural Languages: Papers

2001

pdf bib
Pronominal anaphora resolution in KANTOO English-to-Spanish machine translation system
Teruko Mitamura | Eric Nyberg | Enrique Torrejon | David Svoboda | Kathryn Baker
Proceedings of Machine Translation Summit VIII

We describe the automatic resolution of pronominal anaphora using KANT Controlled English (KCE) and the KANTOO English-to-Spanish MT system. Our algorithm is based on a robust, syntax-based approach that applies a set of restrictions and preferences to select the correct antecedent. We report a success rate of 89.6% on a training corpus with 289 anaphors, and 87.5% on held-out data containing 145 anaphors. Resolution of anaphors is important in translation, due to gender mismatches among languages; our approach translates anaphors to Spanish with 97.2% accuracy.

2000

bib
Controlled languages
Teruko Mitamura | Eric Nyberg
Proceedings of the Fourth Conference of the Association for Machine Translation in the Americas: Tutorial Descriptions

pdf bib
Challenges in adapting an interlingua for bidirectional English-Italian translation
Violetta Cavalli-Sforza | Krzysztof Czuba | Teruko Mitamura | Eric Nyberg
Proceedings of the Fourth Conference of the Association for Machine Translation in the Americas: Technical Papers

We describe our experience in adapting an existing high- quality, interlingual, unidirectional machine translation system to a new domain and bidirectional translation for a new language pair (English and Italian). We focus on the interlingua design changes which were necessary to achieve high quality output in view of the language mismatches between English and Italian. The representation we propose contains features that are interpreted differently, depending on the translation direction. This decision simplified the process of creating the interlingua for individual sentences, and allows the system to defer mapping of language-specific features (such as tense and aspect), which are realized when the target syntactic feature structure is created. We also describe a set of problems we encountered in translating modal verbs, and discuss the representation of modality in our interlingua.

pdf bib
The KANTOO machine translation environment
Eric Nyberg | Teruko Mitamura
Proceedings of the Fourth Conference of the Association for Machine Translation in the Americas: System Descriptions

In this paper we describe the KANTOO machine translation environment, a set of software services and tools for multilingual document production. KANTOO includes modules for source language analysis, target language generation, source terminology management, target terminology management, and knowledge source development. The KANTOOsystem represents a complete re-design and re-implementation of the KANT machine translation system.

pdf bib
Arabic Morphology Generation Using a Concatenative Strategy
Violetta Cavalli-Sforza | Abdelhadi Soudi | Teruko Mitamura
1st Meeting of the North American Chapter of the Association for Computational Linguistics

1999

bib
Controlled language for multilingual machine translation
Teruko Mitamura
Proceedings of Machine Translation Summit VII

pdf bib
Multiple strategies for automatic disambiguation in technical translation
Teruko Mitamura | Eric Nyberg | Enrique Torrejon | Robert Igo
Proceedings of the 8th Conference on Theoretical and Methodological Issues in Machine Translation of Natural Languages

1998

pdf bib
An English-to-Turkish interlingual MT system
Dilek Zeynap Hakkani | Göklan Tür | Kemal Oflazer | Teruko Mitamura | Eric H. Nyberg, 3rd
Proceedings of the Third Conference of the Association for Machine Translation in the Americas: Technical Papers

This paper describes the integration of a Turkish generation system with the KANT knowledge-based machine translation system to produce a prototype English-Turkish interlingua-based machine translation system. These two independently constructed systems were successfully integrated within a period of two months, through development of a module which maps KANT interlingua expressions to Turkish syntactic structures. The combined system is able to translate completely and correctly 44 of 52 benchmark sentences in the domain of broadcast news captions. This study is the first known application of knowledge-based machine translation from English to Turkish, and our initial results show promise for future development.

1997

pdf bib
A Real-Time MT System for Translating Broadcast Captions
Eric Nyberg | Teruko Mitamura
Proceedings of Machine Translation Summit VI: Papers

This presentation demonstrates a new multi-engine machine translation system, which combines knowledge-based and example-based machine translation strategies for real-time translation of business news captions from English to German.

1995

pdf bib
Controlled English for Knowledge-Based MT: Experience with the KANT System
Teruko Mitamura | Eric H. Nyberg 3rd
Proceedings of the Sixth Conference on Theoretical and Methodological Issues in Machine Translation of Natural Languages

1994

pdf bib
Coping With Ambiguity in a Large-Scale Machine Translation System
Kathryn L. Baker | Alexander M. Franz | Pamela W. Jordan | Teruko Mitamura | Eric H. Nyberg
COLING 1994 Volume 1: The 15th International Conference on Computational Linguistics

pdf bib
Evaluation Metrics for Knowledge-Based Machine Translation
Eric H. Nyberg, 3rd | Teruko Mitamura | Jaime G. Carbonell
COLING 1994 Volume 1: The 15th International Conference on Computational Linguistics

pdf bib
KANT: Knowledge-Based, Accurate Natural Language Translation
Teruko Mitamura | Eric Nyberg | Jaime Carbonell
Proceedings of the First Conference of the Association for Machine Translation in the Americas

1993

pdf bib
The TIPSTER/SHOGUN Project
Paul S. Jacobs | George Krupka | Lisa Rau | Michael L. Mauldin | Teruko Mitamura | Tsuyoshi Kitani | Ira Sider | Lois Childs
TIPSTER TEXT PROGRAM: PHASE I: Proceedings of a Workshop held at Fredricksburg, Virginia, September 19-23, 1993

1992

pdf bib
The KANT System: Fast, Accurate, High-Quality Translation in Practical Domains
Eric H. Nyberg III | Teruko Mitamura
COLING 1992 Volume 3: The 14th International Conference on Computational Linguistics

pdf bib
Hierarchical Lexical Structure and Interpretive Mapping in Machine Translation
Teruko Mitamura | Eric H. Nyberg III
COLING 1992 Volume 4: The 14th International Conference on Computational Linguistics

pdf bib
The KANT perspective: a critique of pure transfer (and pure interlingua, pure statistics, .. )
Jaime G. Carbonell | Teruko Mitamura | Eric H. Nyberg 3rd
Proceedings of the Fourth Conference on Theoretical and Methodological Issues in Machine Translation of Natural Languages

1991

pdf bib
An Efficient Interlingua Translation System for Multi-lingual Document Production
Teruko Mitamura | Eric H. Nyberg | Jaime G. Carbonell
Proceedings of Machine Translation Summit III: Papers

Knowledge-based interlingual machine translation systems produce semantically accurate translations, but typically require massive knowledge acquisition. This paper describes KANT, a system that reduces this requirement to produce practical, scalable, and accurate KBMT applications. First, the set of requirements is discussed, then the full KANT architecture is illustrated, and finally results from a fully implemented prototype are presented.

1989

pdf bib
Massively Parallel Parsing in 𝛷DmDialog: Integrated Architecture for Parsing Speech Inputs
Hiroaki Kitano | Teruko Mitamura | Masaru Tomita
Proceedings of the First International Workshop on Parsing Technologies

This paper describes the parsing scheme in the 𝛷DmDialog speech-to-speech dialog translation system, with special emphasis on the integration of speech and natural language processing. We propose an integrated architecture for parsing speech inputs based on a parallel marker-passing scheme and attaining dynamic participation of knowledge from the phonological-level to the discourse-level. At the phonological level, we employ a stochastic model using a transition matrix and a confusion matrix and markers which carry a probability measure. At a higher level, syntactic/semantic and discourse processing, we integrate a case-based and constraint-based scheme in a consistent manner so that a priori probability and constraints, which reflect linguistic and discourse factors, are provided to the phonological level of processing. A probability/cost-based scheme in our model enables ambiguity resolution at various levels using one uniform principle.

1988

pdf bib
The Universal Parser Compiler and its application to a speech translation system
Masaru Tomita | Marion Kee | Hiroaki Saito | Teruko Mitamura | Hideto Tomabechi
Proceedings of the Second Conference on Theoretical and Methodological Issues in Machine Translation of Natural Languages

Search
Co-authors