Wolfgang Maier


2021

pdf bib
Blending Task Success and User Satisfaction: Analysis of Learned Dialogue Behaviour with Multiple Rewards
Stefan Ultes | Wolfgang Maier
Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue

Recently, principal reward components for dialogue policy reinforcement learning use task success and user satisfaction independently and neither the resulting learned behaviour has been analysed nor a suitable proper analysis method even existed. In this work, we employ both principal reward components jointly and propose a method to analyse the resulting behaviour through a structured way of probing the learned policy. We show that blending both reward components increases user satisfaction without sacrificing task success in more hostile environments and provide insight about actions chosen by the learned policies.

pdf bib
Naturalness Evaluation of Natural Language Generation in Task-oriented Dialogues Using BERT
Ye Liu | Wolfgang Maier | Wolfgang Minker | Stefan Ultes
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)

This paper presents an automatic method to evaluate the naturalness of natural language generation in dialogue systems. While this task was previously rendered through expensive and time-consuming human labor, we present this novel task of automatic naturalness evaluation of generated language. By fine-tuning the BERT model, our proposed naturalness evaluation method shows robust results and outperforms the baselines: support vector machines, bi-directional LSTMs, and BLEURT. In addition, the training speed and evaluation performance of naturalness model are improved by transfer learning from quality and informativeness linguistic knowledge.

2020

pdf bib
Similarity Scoring for Dialogue Behaviour Comparison
Stefan Ultes | Wolfgang Maier
Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue

The differences in decision making between behavioural models of voice interfaces are hard to capture using existing measures for the absolute performance of such models. For instance, two models may have a similar task success rate, but very different ways of getting there. In this paper, we propose a general methodology to compute the similarity of two dialogue behaviour models and investigate different ways of computing scores on both the semantic and the textual level. Complementing absolute measures of performance, we test our scores on three different tasks and show the practical usability of the measures.

2018

pdf bib
Towards an Automatic Assessment of Crowdsourced Data for NLU
Patricia Braunger | Wolfgang Maier | Jan Wessling | Maria Schmidt
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)

pdf bib
GHHT at CALCS 2018: Named Entity Recognition for Dialectal Arabic Using Neural Networks
Mohammed Attia | Younes Samih | Wolfgang Maier
Proceedings of the Third Workshop on Computational Approaches to Linguistic Code-Switching

This paper describes our system submission to the CALCS 2018 shared task on named entity recognition on code-switched data for the language variant pair of Modern Standard Arabic and Egyptian dialectal Arabic. We build a a Deep Neural Network that combines word and character-based representations in convolutional and recurrent networks with a CRF layer. The model is augmented with stacked layers of enriched information such pre-trained embeddings, Brown clusters and named entity gazetteers. Our system is ranked second among those participating in the shared task achieving an FB1 average of 70.09%.

pdf bib
GHH at SemEval-2018 Task 10: Discovering Discriminative Attributes in Distributional Semantics
Mohammed Attia | Younes Samih | Manaal Faruqui | Wolfgang Maier
Proceedings of The 12th International Workshop on Semantic Evaluation

This paper describes our system submission to the SemEval 2018 Task 10 on Capturing Discriminative Attributes. Given two concepts and an attribute, the task is to determine whether the attribute is semantically related to one concept and not the other. In this work we assume that discriminative attributes can be detected by discovering the association (or lack of association) between a pair of words. The hypothesis we test in this contribution is whether the semantic difference between two pairs of concepts can be treated in terms of measuring the distance between words in a vector space, or can simply be obtained as a by-product of word co-occurrence counts.

2017

pdf bib
Natural Language Input for In-Car Spoken Dialog Systems: How Natural is Natural?
Patricia Braunger | Wolfgang Maier
Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue

Recent spoken dialog systems are moving away from command and control towards a more intuitive and natural style of interaction. In order to choose an appropriate system design which allows the system to deal with naturally spoken user input, a definition of what exactly constitutes naturalness in user input is important. In this paper, we examine how different user groups naturally speak to an automotive spoken dialog system (SDS). We conduct a user study in which we collect freely spoken user utterances for a wide range of use cases in German. By means of a comparative study of the utterances from the study with interpersonal utterances, we provide criteria what constitutes naturalness in the user input of an state-of-the-art automotive SDS.

2016

pdf bib
An Arabic-Moroccan Darija Code-Switched Corpus
Younes Samih | Wolfgang Maier
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)

In this paper, we describe our effort in the development and annotation of a large scale corpus containing code-switched data. Until recently, very limited effort has been devoted to develop computational approaches or even basic linguistic resources to support research into the processing of Moroccan Darija.

pdf bib
Proceedings of the Workshop on Discontinuous Structures in Natural Language Processing
Wolfgang Maier | Sandra Kübler | Constantin Orasan
Proceedings of the Workshop on Discontinuous Structures in Natural Language Processing

pdf bib
Discontinuous parsing with continuous trees
Wolfgang Maier | Timm Lichte
Proceedings of the Workshop on Discontinuous Structures in Natural Language Processing

pdf bib
SAWT: Sequence Annotation Web Tool
Younes Samih | Wolfgang Maier | Laura Kallmeyer
Proceedings of the Second Workshop on Computational Approaches to Code Switching

2015

pdf bib
LR Parsing for LCFRS
Laura Kallmeyer | Wolfgang Maier
Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

pdf bib
Discontinuous Incremental Shift-reduce Parsing
Wolfgang Maier
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)

2014

pdf bib
The IUCL+ System: Word-Level Language Identification via Extended Markov Models
Levi King | Eric Baucom | Timur Gilmanov | Sandra Kübler | Dan Whyatt | Wolfgang Maier | Paul Rodrigues
Proceedings of the First Workshop on Computational Approaches to Code Switching

pdf bib
Language variety identification in Spanish tweets
Wolfgang Maier | Carlos Gómez-Rodríguez
Proceedings of the EMNLP’2014 Workshop on Language Technology for Closely Related Languages and Language Variants

pdf bib
Parsing German: How Much Morphology Do We Need?
Wolfgang Maier | Sandra Kübler | Daniel Dakota | Daniel Whyatt
Proceedings of the First Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages

pdf bib
Discosuite - A parser test suite for German discontinuous structures
Wolfgang Maier | Miriam Kaeshammer | Peter Baumann | Sandra Kübler
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)

Parser evaluation traditionally relies on evaluation metrics which deliver a single aggregate score over all sentences in the parser output, such as PARSEVAL. However, for the evaluation of parser performance concerning a particular phenomenon, a test suite of sentences is needed in which this phenomenon has been identified. In recent years, the parsing of discontinuous structures has received a rising interest. Therefore, in this paper, we present a test suite for testing the performance of dependency and constituency parsers on non-projective dependencies and discontinuous constituents for German. The test suite is based on the newly released TIGER treebank version 2.2. It provides a unique possibility of benchmarking parsers on non-local syntactic relationships in German, for constituents and dependencies. We include a linguistic analysis of the phenomena that cause discontinuity in the TIGER annotation, thereby closing gaps in previous literature. The linguistic phenomena we investigate include extraposition, a placeholder/repeated element construction, topicalization, scrambling, local movement, parentheticals, and fronting of pronouns.

2013

pdf bib
Overview of the SPMRL 2013 Shared Task: A Cross-Framework Evaluation of Parsing Morphologically Rich Languages
Djamé Seddah | Reut Tsarfaty | Sandra Kübler | Marie Candito | Jinho D. Choi | Richárd Farkas | Jennifer Foster | Iakes Goenaga | Koldo Gojenola Galletebeitia | Yoav Goldberg | Spence Green | Nizar Habash | Marco Kuhlmann | Wolfgang Maier | Joakim Nivre | Adam Przepiórkowski | Ryan Roth | Wolfgang Seeker | Yannick Versley | Veronika Vincze | Marcin Woliński | Alina Wróblewska | Eric Villemonte de la Clergerie
Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages

pdf bib
LCFRS binarization and debinarization for directional parsing
Wolfgang Maier
Proceedings of the 13th International Conference on Parsing Technologies (IWPT 2013)

pdf bib
Data-Driven Parsing using Probabilistic Linear Context-Free Rewriting Systems
Laura Kallmeyer | Wolfgang Maier
Computational Linguistics, Volume 39, Issue 1 - March 2013

2012

pdf bib
Annotating Coordination in the Penn Treebank
Wolfgang Maier | Sandra Kübler | Erhard Hinrichs | Julia Krivanek
Proceedings of the Sixth Linguistic Annotation Workshop

pdf bib
PLCFRS Parsing Revisited: Restricting the Fan-Out to Two
Wolfgang Maier | Miriam Kaeshammer | Laura Kallmeyer
Proceedings of the 11th International Workshop on Tree Adjoining Grammars and Related Formalisms (TAG+11)

2010

pdf bib
Data-Driven Parsing with Probabilistic Linear Context-Free Rewriting Systems
Laura Kallmeyer | Wolfgang Maier
Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)

pdf bib
Direct Parsing of Discontinuous Constituents in German
Wolfgang Maier
Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages

pdf bib
Discontinuity and Non-Projectivity: Using Mildly Context-Sensitive Formalisms for Data-Driven Parsing
Wolfgang Maier | Laura Kallmeyer
Proceedings of the 10th International Workshop on Tree Adjoining Grammar and Related Frameworks (TAG+10)

2009

pdf bib
An Earley Parsing Algorithm for Range Concatenation Grammars
Laura Kallmeyer | Wolfgang Maier | Yannick Parmentier
Proceedings of the ACL-IJCNLP 2009 Conference Short Papers

pdf bib
Un Algorithme d’Analyse de Type Earley pour Grammaires à Concaténation d’Intervalles
Laura Kallmeyer | Wolfgang Maier | Yannick Parmentier
Actes de la 16ème conférence sur le Traitement Automatique des Langues Naturelles. Articles courts

Nous présentons ici différents algorithmes d’analyse pour grammaires à concaténation d’intervalles (Range Concatenation Grammar, RCG), dont un nouvel algorithme de type Earley, dans le paradigme de l’analyse déductive. Notre travail est motivé par l’intérêt porté récemment à ce type de grammaire, et comble un manque dans la littérature existante.

pdf bib
Parsing Coordinations
Sandra Kübler | Erhard Hinrichs | Wolfgang Maier | Eva Klett
Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009)

pdf bib
An Incremental Earley Parser for Simple Range Concatenation Grammar
Laura Kallmeyer | Wolfgang Maier
Proceedings of the 11th International Conference on Parsing Technologies (IWPT’09)

pdf bib
Synchronous Rewriting in Treebanks
Laura Kallmeyer | Wolfgang Maier | Giorgio Satta
Proceedings of the 11th International Conference on Parsing Technologies (IWPT’09)

2008

pdf bib
Proceedings of the ACL-08: HLT Student Research Workshop
Ebru Arisoy | Keisuke Inoue | Wolfgang Maier
Proceedings of the ACL-08: HLT Student Research Workshop

pdf bib
Developing a TT-MCTAG for German with an RCG-based Parser
Laura Kallmeyer | Timm Lichte | Wolfgang Maier | Yannick Parmentier | Johannes Dellert
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)

Developing linguistic resources, in particular grammars, is known to be a complex task in itself, because of (amongst others) redundancy and consistency issues. Furthermore some languages can reveal themselves hard to describe because of specific characteristics, e.g. the free word order in German. In this context, we present (i) a framework allowing to describe tree-based grammars, and (ii) an actual fragment of a core multicomponent tree-adjoining grammar with tree tuples (TT-MCTAG) for German developed using this framework. This framework combines a metagrammar compiler and a parser based on range concatenation grammar (RCG) to respectively check the consistency and the correction of the grammar. The German grammar being developed within this framework already deals with a wide range of scrambling and extraction phenomena.

pdf bib
How to Compare Treebanks
Sandra Kübler | Wolfgang Maier | Ines Rehbein | Yannick Versley
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)

Recent years have seen an increasing interest in developing standards for linguistic annotation, with a focus on the interoperability of the resources. This effort, however, requires a profound knowledge of the advantages and disadvantages of linguistic annotation schemes in order to avoid importing the flaws and weaknesses of existing encoding schemes into the new standards. This paper addresses the question how to compare syntactically annotated corpora and gain insights into the usefulness of specific design decisions. We present an exhaustive evaluation of two German treebanks with crucially different encoding schemes. We evaluate three different parsers trained on the two treebanks and compare results using EvalB, the Leaf-Ancestor metric, and a dependency-based evaluation. Furthermore, we present TePaCoC, a new testsuite for the evaluation of parsers on complex German grammatical constructions. The testsuite provides a well thought-out error classification, which enables us to compare parser output for parsers trained on treebanks with different encoding schemes and provides interesting insights into the impact of treebank annotation schemes on specific constructions like PP attachment or non-constituent coordination.

pdf bib
TuLiPA: Towards a Multi-Formalism Parsing Environment for Grammar Engineering
Laura Kallmeyer | Timm Lichte | Wolfgang Maier | Yannick Parmentier | Johannes Dellert | Kilian Evang
Coling 2008: Proceedings of the workshop on Grammar Engineering Across Frameworks

pdf bib
TuLiPA: A syntax-semantics parsing environment for mildly context-sensitive formalisms
Yannick Parmentier | Laura Kallmeyer | Wolfgang Maier | Timm Lichte | Johannes Dellert
Proceedings of the Ninth International Workshop on Tree Adjoining Grammar and Related Frameworks (TAG+9)

2006

pdf bib
Is it Really that Difficult to Parse German?
Sandra Kübler | Erhard W. Hinrichs | Wolfgang Maier
Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing

pdf bib
Annotation Schemes and their Influence on Parsing Results
Wolfgang Maier
Proceedings of the COLING/ACL 2006 Student Research Workshop