Roy Eisenstadt


pdf bib
An End-to-End Dialogue Summarization System for Sales Calls
Abedelkadir Asi | Song Wang | Roy Eisenstadt | Dean Geckt | Yarin Kuper | Yi Mao | Royi Ronen
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track

Summarizing sales calls is a routine task performed manually by salespeople. We present a production system which combines generative models fine-tuned for customer-agent setting, with a human-in-the-loop user experience for an interactive summary curation process. We address challenging aspects of dialogue summarization task in a real-world setting including long input dialogues, content validation, lack of labeled data and quality evaluation. We show how GPT-3 can be leveraged as an offline data labeler to handle training data scarcity and accommodate privacy constraints in an industrial setting. Experiments show significant improvements by our models in tackling the summarization and content validation tasks on public datasets.


pdf bib
Evaluation Guidelines to Deal with Implicit Phenomena to Assess Factuality in Data-to-Text Generation
Roy Eisenstadt | Michael Elhadad
Proceedings of the 1st Workshop on Understanding Implicit and Underspecified Language

Data-to-text generation systems are trained on large datasets, such as WebNLG, Ro-toWire, E2E or DART. Beyond traditional token-overlap evaluation metrics (BLEU or METEOR), a key concern faced by recent generators is to control the factuality of the generated text with respect to the input data specification. We report on our experience when developing an automatic factuality evaluation system for data-to-text generation that we are testing on WebNLG and E2E data. We aim to prepare gold data annotated manually to identify cases where the text communicates more information than is warranted based on the in-put data (extra) or fails to communicate data that is part of the input (missing). While analyzing reference (data, text) samples, we encountered a range of systematic uncertainties that are related to cases on implicit phenomena in text, and the nature of non-linguistic knowledge we expect to be involved when assessing factuality. We derive from our experience a set of evaluation guidelines to reach high inter-annotator agreement on such cases.


pdf bib
Neural Micro-Planning for Data to Text Generation Produces more Cohesive Text
Roy Eisenstadt | Michael Elhadad
Proceedings of the Workshop on Discourse Theories for Text Planning