Michael Elhadad


2021

pdf bib
Evaluation Guidelines to Deal with Implicit Phenomena to Assess Factuality in Data-to-Text Generation
Roy Eisenstadt | Michael Elhadad
Proceedings of the 1st Workshop on Understanding Implicit and Underspecified Language

Data-to-text generation systems are trained on large datasets, such as WebNLG, Ro-toWire, E2E or DART. Beyond traditional token-overlap evaluation metrics (BLEU or METEOR), a key concern faced by recent generators is to control the factuality of the generated text with respect to the input data specification. We report on our experience when developing an automatic factuality evaluation system for data-to-text generation that we are testing on WebNLG and E2E data. We aim to prepare gold data annotated manually to identify cases where the text communicates more information than is warranted based on the in-put data (extra) or fails to communicate data that is part of the input (missing). While analyzing reference (data, text) samples, we encountered a range of systematic uncertainties that are related to cases on implicit phenomena in text, and the nature of non-linguistic knowledge we expect to be involved when assessing factuality. We derive from our experience a set of evaluation guidelines to reach high inter-annotator agreement on such cases.

pdf bib
Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA
Yonatan Bitton | Gabriel Stanovsky | Roy Schwartz | Michael Elhadad
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Recent works have shown that supervised models often exploit data artifacts to achieve good test scores while their performance severely degrades on samples outside their training distribution. Contrast sets (Gardneret al., 2020) quantify this phenomenon by perturbing test samples in a minimal way such that the output label is modified. While most contrast sets were created manually, requiring intensive annotation effort, we present a novel method which leverages rich semantic input representation to automatically generate contrast sets for the visual question answering task. Our method computes the answer of perturbed questions, thus vastly reducing annotation cost and enabling thorough evaluation of models’ performance on various semantic aspects (e.g., spatial or relational reasoning). We demonstrate the effectiveness of our approach on the GQA dataset and its semantic scene graph image representation. We find that, despite GQA’s compositionality and carefully balanced label distribution, two high-performing models drop 13-17% in accuracy compared to the original test set. Finally, we show that our automatic perturbation can be applied to the training set to mitigate the degradation in performance, opening the door to more robust models.

2020

pdf bib
Building a Hebrew Semantic Role Labeling Lexical Resource from Parallel Movie Subtitles
Ben Eyal | Michael Elhadad
Proceedings of the 12th Language Resources and Evaluation Conference

We present a semantic role labeling resource for Hebrew built semi-automatically through annotation projection from English. This corpus is derived from the multilingual OpenSubtitles dataset and includes short informal sentences, for which reliable linguistic annotations have been computed. We provide a fully annotated version of the data including morphological analysis, dependency syntax and semantic role labeling in both FrameNet and ProbBank styles. Sentences are aligned between English and Hebrew, both sides include full annotations and the explicit mapping from the English arguments to the Hebrew ones. We train a neural SRL model on this Hebrew resource exploiting the pre-trained multilingual BERT transformer model, and provide the first available baseline model for Hebrew SRL as a reference point. The code we provide is generic and can be adapted to other languages to bootstrap SRL resources.

pdf bib
Neural Micro-Planning for Data to Text Generation Produces more Cohesive Text
Roy Eisenstadt | Michael Elhadad
Proceedings of the Workshop on Discourse Theories for Text Planning

2019

pdf bib
Question Answering as an Automatic Evaluation Metric for News Article Summarization
Matan Eyal | Tal Baumel | Michael Elhadad
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)

Recent work in the field of automatic summarization and headline generation focuses on maximizing ROUGE scores for various news datasets. We present an alternative, extrinsic, evaluation metric for this task, Answering Performance for Evaluation of Summaries. APES utilizes recent progress in the field of reading-comprehension to quantify the ability of a summary to answer a set of manually created questions regarding central entities in the source article. We first analyze the strength of this metric by comparing it to known manual evaluation metrics. We then present an end-to-end neural abstractive model that maximizes APES, while increasing ROUGE scores to competitive results.

2016

pdf bib
The Hebrew FrameNet Project
Avi Hayoun | Michael Elhadad
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)

We present the Hebrew FrameNet project, describe the development and annotation processes and enumerate the challenges we faced along the way. We have developed semi-automatic tools to help speed the annotation and data collection process. The resource currently covers 167 frames, 3,000 lexical units and about 500 fully annotated sentences. We have started training and testing automatic SRL tools on the seed data.

pdf bib
Sentence Embedding Evaluation Using Pyramid Annotation
Tal Baumel | Raphael Cohen | Michael Elhadad
Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP

2014

pdf bib
Query-Chain Focused Summarization
Tal Baumel | Raphael Cohen | Michael Elhadad
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

2013

pdf bib
Effect of Out Of Vocabulary Terms on Inferring Eligibility Criteria for a Retrospective Study in Hebrew EHR
Raphael Cohen | Michael Elhadad
Proceedings of the 2013 Workshop on Biomedical Natural Language Processing

pdf bib
Multi-document multilingual summarization corpus preparation, Part 2: Czech, Hebrew and Spanish
Michael Elhadad | Sabino Miranda-Jiménez | Josef Steinberger | George Giannakopoulos
Proceedings of the MultiLing 2013 Workshop on Multilingual Multi-document Summarization

pdf bib
Word Segmentation, Unknown-word Resolution, and Morphological Agreement in a Hebrew Parsing System
Yoav Goldberg | Michael Elhadad
Computational Linguistics, Volume 39, Issue 1 - March 2013

2012

pdf bib
Domain Adaptation of a Dependency Parser with a Class-Class Selectional Preference Model
Raphael Cohen | Yoav Goldberg | Michael Elhadad
Proceedings of ACL 2012 Student Research Workshop

2011

pdf bib
Joint Hebrew Segmentation and Parsing using a PCFGLA Lattice Parser
Yoav Goldberg | Michael Elhadad
Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies

2010

pdf bib
An Efficient Algorithm for Easy-First Non-Directional Dependency Parsing
Yoav Goldberg | Michael Elhadad
Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics

pdf bib
Easy-First Dependency Parsing of Modern Hebrew
Yoav Goldberg | Michael Elhadad
Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages

pdf bib
Inspecting the Structural Biases of Dependency Parsing Algorithms
Yoav Goldberg | Michael Elhadad
Proceedings of the Fourteenth Conference on Computational Natural Language Learning

pdf bib
Book Review: Natural Language Processing with Python by Steven Bird, Ewan Klein, and Edward Loper
Michael Elhadad
Computational Linguistics, Volume 36, Issue 4 - December 2010

2009

pdf bib
Enhancing Unlexicalized Parsing Performance Using a Wide Coverage Lexicon, Fuzzy Tag-Set Mapping, and EM-HMM-Based Lexical Probabilities
Yoav Goldberg | Reut Tsarfaty | Meni Adler | Michael Elhadad
Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009)

pdf bib
On the Role of Lexical Features in Sequence Labeling
Yoav Goldberg | Michael Elhadad
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

pdf bib
Gaiku : Generating Haiku with Word Associations Norms
Yael Netzer | David Gabay | Yoav Goldberg | Michael Elhadad
Proceedings of the Workshop on Computational Approaches to Linguistic Creativity

pdf bib
Hebrew Dependency Parsing: Initial Results
Yoav Goldberg | Michael Elhadad
Proceedings of the 11th International Conference on Parsing Technologies (IWPT’09)

2008

pdf bib
Tagging a Hebrew Corpus: the Case of Participles
Meni Adler | Yael Netzer | Yoav Goldberg | David Gabay | Michael Elhadad
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)

We report on an effort to build a corpus of Modern Hebrew tagged with part-of-speech and morphology. We designed a tagset specific to Hebrew while focusing on four aspects: the tagset should be consistent with common linguistic knowledge; there should be maximal agreement among taggers as to the tags assigned to maintain consistency; the tagset should be useful for machine taggers and learning algorithms; and the tagset should be effective for applications relying on the tags as input features. In this paper, we illustrate these issues by explaining our decision to introduce a tag for beinoni forms in Hebrew. We explain how this tag is defined, and how it helped us improve manual tagging accuracy to a high-level, while improving automatic tagging and helping in the task of syntactic chunking.

pdf bib
Unsupervised Lexicon-Based Resolution of Unknown Words for Full Morphological Analysis
Meni Adler | Yoav Goldberg | David Gabay | Michael Elhadad
Proceedings of ACL-08: HLT

pdf bib
EM Can Find Pretty Good HMM POS-Taggers (When Given a Good Start)
Yoav Goldberg | Meni Adler | Michael Elhadad
Proceedings of ACL-08: HLT

pdf bib
splitSVM: Fast, Space-Efficient, non-Heuristic, Polynomial Kernel Computation for NLP Applications
Yoav Goldberg | Michael Elhadad
Proceedings of ACL-08: HLT, Short Papers

2007

pdf bib
Can You Tag the Modal? You Should.
Yael Netzer | Meni Adler | David Gabay | Michael Elhadad
Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources

pdf bib
SVM Model Tampering and Anchored Learning: A Case Study in Hebrew NP Chunking
Yoav Goldberg | Michael Elhadad
Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics

2006

pdf bib
An Unsupervised Morpheme-Based HMM for Hebrew Morphological Disambiguation
Meni Adler | Michael Elhadad
Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics

pdf bib
Noun Phrase Chunking in Hebrew: Influence of Lexical and Morphological Features
Yoav Goldberg | Meni Adler | Michael Elhadad
Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics

pdf bib
Using Semantic Authoring for Blissymbols Communication Boards
Yael Netzer | Michael Elhadad
Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers

2005

pdf bib
Interactive Authoring of Logical Forms for Multilingual Generation
Ofer Biller | Michael Elhadad | Yael Netzer
Proceedings of the Tenth European Workshop on Natural Language Generation (ENLG-05)

2000

pdf bib
INLG’2000 Proceedings of the First International Conference on Natural Language Generation
Michael Elhadad
INLG’2000 Proceedings of the First International Conference on Natural Language Generation

pdf bib
Integrating a Large-Scale, Reusable Lexicon with a Natural Language Generator
Hongyan Jing | Yael Dahan | Michael Elhadad | Kathy McKeown
INLG’2000 Proceedings of the First International Conference on Natural Language Generation

1999

pdf bib
Bilingual Hebrew-English Generation of Possessives and Partitives: Raising the Input Abstraction Level
Yael Dahan Netzer | Michael Elhadad
Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics

pdf bib
Information Fusion in the Context of Multi-Document Summarization
Regina Barzilay | Kathleen R. McKeown | Michael Elhadad
Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics

1998

pdf bib
Generating Determiners and Quantifiers in Hebrew
Yael Dahan Netzer | Michael Elhadad
Computational Approaches to Semitic Languages

pdf bib
Generation of Noun Compounds in Hebrew: Can Syntactic Knowledge Be Fully Encapsulated?
Yael Dahan Netzer | Michael Elhadad
Natural Language Generation

1997

pdf bib
Floating Constraints in Lexical Choice
Michael Elhadad | Kathleen McKeown | Jacques Robin
Computational Linguistics, Volume 23, Number 2, June 1997

pdf bib
Using Lexical Chains for Text Summarization
Regina Barzilay | Michael Elhadad
Intelligent Scalable Text Summarization

1996

pdf bib
An Overview of SURGE: a Reusable Comprehensive Syntactic Realization Component
Michael Elhadad | Jacques Robin
Eighth International Natural Language Generation Workshop (Posters and Demonstrations)

1992

pdf bib
Generating Coherent Argumentative Paragraphs
Michael Elhadad
COLING 1992 Volume 2: The 14th International Conference on Computational Linguistics

1990

pdf bib
Types in Functional Unification Grammars
Michael Elhadad
28th Annual Meeting of the Association for Computational Linguistics

pdf bib
Generating Connectives
Michael Elhadad | Kathleen R. McKeown
COLING 1990 Volume 3: Papers presented to the 13th International Conference on Computational Linguistics