Xavier Carreras


2021

pdf bib
Minimizing Annotation Effort via Max-Volume Spectral Sampling
Ariadna Quattoni | Xavier Carreras
Findings of the Association for Computational Linguistics: EMNLP 2021

We address the annotation data bottleneck for sequence classification. Specifically we ask the question: if one has a budget of N annotations, which samples should we select for annotation? The solution we propose looks for diversity in the selected sample, by maximizing the amount of information that is useful for the learning algorithm, or equivalently by minimizing the redundancy of samples in the selection. This is formulated in the context of spectral learning of recurrent functions for sequence classification. Our method represents unlabeled data in the form of a Hankel matrix, and uses the notion of spectral max-volume to find a compact sub-block from which annotation samples are drawn. Experiments on sequence classification confirm that our spectral sampling strategy is in fact efficient and yields good models.

2020

pdf bib
A comparison between CNNs and WFAs for Sequence Classification
Ariadna Quattoni | Xavier Carreras
Proceedings of SustaiNLP: Workshop on Simple and Efficient Natural Language Processing

We compare a classical CNN architecture for sequence classification involving several convolutional and max-pooling layers against a simple model based on weighted finite state automata (WFA). Each model has its advantages and disadvantages and it is possible that they could be combined. However, we believe that the first research goal should be to investigate and understand how do these two apparently dissimilar models compare in the context of specific natural language processing tasks. This paper is the first step towards that goal. Our experiments with five sequence classification datasets suggest that, despite the apparent simplicity of WFA models and training algorithms, the performance of WFAs is comparable to that of the CNNs.

2019

pdf bib
Interpolated Spectral NGram Language Models
Ariadna Quattoni | Xavier Carreras
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Spectral models for learning weighted non-deterministic automata have nice theoretical and algorithmic properties. Despite this, it has been challenging to obtain competitive results in language modeling tasks, for two main reasons. First, in order to capture long-range dependencies of the data, the method must use statistics from long substrings, which results in very large matrices that are difficult to decompose. The second is that the loss function behind spectral learning, based on moment matching, differs from the probabilistic metrics used to evaluate language models. In this work we employ a technique for scaling up spectral learning, and use interpolated predictions that are optimized to maximize perplexity. Our experiments in character-based language modeling show that our method matches the performance of state-of-the-art ngram models, while being very fast to train.

2018

pdf bib
Local String Transduction as Sequence Labeling
Joana Ribeiro | Shashi Narayan | Shay B. Cohen | Xavier Carreras
Proceedings of the 27th International Conference on Computational Linguistics

We show that the general problem of string transduction can be reduced to the problem of sequence labeling. While character deletion and insertions are allowed in string transduction, they do not exist in sequence labeling. We show how to overcome this difference. Our approach can be used with any sequence labeling algorithm and it works best for problems in which string transduction imposes a strong notion of locality (no long range dependencies). We experiment with spelling correction for social media, OCR correction, and morphological inflection, and we see that it behaves better than seq2seq models and yields state-of-the-art results in several cases.

2017

pdf bib
Prepositional Phrase Attachment over Word Embedding Products
Pranava Swaroop Madhyastha | Xavier Carreras | Ariadna Quattoni
Proceedings of the 15th International Conference on Parsing Technologies

We present a low-rank multi-linear model for the task of solving prepositional phrase attachment ambiguity (PP task). Our model exploits tensor products of word embeddings, capturing all possible conjunctions of latent embeddings. Our results on a wide range of datasets and task settings show that tensor products are the best compositional operation and that a relatively simple multi-linear model that uses only word embeddings of lexical features can outperform more complex non-linear architectures that exploit the same information. Our proposed model gives the current best reported performance on an out-of-domain evaluation and performs competively on out-of-domain dependency parsing datasets.

pdf bib
Arc-Standard Spinal Parsing with Stack-LSTMs
Miguel Ballesteros | Xavier Carreras
Proceedings of the 15th International Conference on Parsing Technologies

We present a neural transition-based parser for spinal trees, a dependency representation of constituent trees. The parser uses Stack-LSTMs that compose constituent nodes with dependency-based derivations. In experiments, we show that this model adapts to different styles of dependency relations, but this choice has little effect for predicting constituent structure, suggesting that LSTMs induce useful states by themselves.

2016

pdf bib
Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
Jian Su | Kevin Duh | Xavier Carreras
Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing

2015

pdf bib
Low-Rank Regularization for Sparse Conjunctive Feature Spaces: An Application to Named Entity Classification
Audi Primadhanty | Xavier Carreras | Ariadna Quattoni
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)

pdf bib
Named entity recognition with document-specific KB tag gazetteers
Will Radford | Xavier Carreras | James Henderson
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

pdf bib
Transition-based Spinal Parsing
Miguel Ballesteros | Xavier Carreras
Proceedings of the Nineteenth Conference on Computational Natural Language Learning

2014

pdf bib
XLike Project Language Analysis Services
Xavier Carreras | Lluís Padró | Lei Zhang | Achim Rettinger | Zhixing Li | Esteban García-Cuesta | Željko Agić | Božo Bekavac | Blaz Fortuna | Tadej Štajner
Proceedings of the Demonstrations at the 14th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
Learning Task-specific Bilexical Embeddings
Pranava Swaroop Madhyastha | Xavier Carreras | Ariadna Quattoni
Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers

pdf bib
Language Processing Infrastructure in the XLike Project
Lluís Padró | Željko Agić | Xavier Carreras | Blaz Fortuna | Esteban García-Cuesta | Zhixing Li | Tadej Štajner | Marko Tadić
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)

This paper presents the linguistic analysis tools and its infrastructure developed within the XLike project. The main goal of the implemented tools is to provide a set of functionalities for supporting some of the main objectives of XLike, such as enabling cross-lingual services for publishers, media monitoring or developing new business intelligence applications. The services cover seven major and minor languages: English, German, Spanish, Chinese, Catalan, Slovenian, and Croatian. These analyzers are provided as web services following a lightweight SOA architecture approach, and they are publically callable and are catalogued in META-SHARE.

pdf bib
Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation
Dekai Wu | Marine Carpuat | Xavier Carreras | Eva Maria Vecchi
Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation

pdf bib
A Shortest-path Method for Arc-factored Semantic Role Labeling
Xavier Lluís | Xavier Carreras | Lluís Màrquez
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)

bib
Spectral Learning Techniques for Weighted Automata, Transducers, and Grammars
Borja Balle | Ariadna Quattoni | Xavier Carreras
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts

In recent years we have seen the development of efficient and provably correct algorithms for learning weighted automata and closely related function classes such as weighted transducers and weighted context-free grammars. The common denominator of all these algorithms is the so-called spectral method, which gives an efficient and robust way to estimate recursively defined functions from empirical estimations of observable statistics. These algorithms are appealing because of the existence of theoretical guarantees (e.g. they are not susceptible to local minima) and because of their efficiency. However, despite their simplicity and wide applicability to real problems, their impact in NLP applications is still moderate. One of the goals of this tutorial is to remedy this situation.The contents that will be presented in this tutorial will offer a complementary perspective with respect to previous tutorials on spectral methods presented at ICML-2012, ICML-2013 and NAACL-2013. Rather than using the language of graphical models and signal processing, we tell the story from the perspective of formal languages and automata theory (without assuming a background in formal algebraic methods). Our presentation highlights the common intuitions lying behind different spectral algorithms by presenting them in a unified framework based on the concepts of low-rank factorizations and completions of Hankel matrices. In addition, we provide an interpretation of the method in terms of forward and backward recursions for automata and grammars. This provides extra intuitions about the method and stresses the importance of matrix factorization for learning automata and grammars. We believe that this complementary perspective might be appealing for an NLP audience and serve to put spectral learning in a wider and, perhaps for some, more familiar context. Our hope is that this will broaden the understanding of these methods by the NLP community and empower many researchers to apply these techniques to novel problems.The content of the tutorial will be divided into four blocks of 45 minutes each, as follows. The first block will introduce the basic definitions of weighted automata and Hankel matrices, and present a key connection between the fundamental theorem of weighted automata and learning. In the second block we will discuss the case of probabilistic automata in detail, touching upon all aspects from the underlying theory to the tricks required to achieve accurate and scalable learning algorithms. The third block will present extensions to related models, including sequence tagging models, finite-state transducers and weighted context-free grammars. The last block will describe a general framework for using spectral techniques in more general situations where a matrix completion pre-processing step is required; several applications of this approach will be described.

2013

pdf bib
Unsupervised Spectral Learning of WCFG as Low-rank Matrix Completion
Raphaël Bailly | Xavier Carreras | Franco M. Luque | Ariadna Quattoni
Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing

pdf bib
Joint Arc-factored Parsing of Syntactic and Semantic Dependencies
Xavier Lluís | Xavier Carreras | Lluís Màrquez
Transactions of the Association for Computational Linguistics, Volume 1

In this paper we introduce a joint arc-factored model for syntactic and semantic dependency parsing. The semantic role labeler predicts the full syntactic paths that connect predicates with their arguments. This process is framed as a linear assignment task, which allows to control some well-formedness constraints. For the syntactic part, we define a standard arc-factored dependency model that predicts the full syntactic tree. Finally, we employ dual decomposition techniques to produce consistent syntactic and predicate-argument structures while searching over a large space of syntactic configurations. In experiments on the CoNLL-2009 English benchmark we observe very competitive results.

2012

pdf bib
Spectral Learning for Non-Deterministic Dependency Parsing
Franco M. Luque | Ariadna Quattoni | Borja Balle | Xavier Carreras
Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics

2009

pdf bib
Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009)
Suzanne Stevenson | Xavier Carreras
Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009)

pdf bib
Non-Projective Parsing for Statistical Machine Translation
Xavier Carreras | Michael Collins
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

pdf bib
An Empirical Study of Semi-supervised Structured Conditional Models for Dependency Parsing
Jun Suzuki | Hideki Isozaki | Xavier Carreras | Michael Collins
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

2008

pdf bib
TAG, Dynamic Programming, and the Perceptron for Efficient, Feature-Rich Parsing
Xavier Carreras | Michael Collins | Terry Koo
CoNLL 2008: Proceedings of the Twelfth Conference on Computational Natural Language Learning

pdf bib
Special Issue Introduction: Semantic Role Labeling: An Introduction to the Special Issue
Lluís Màrquez | Xavier Carreras | Kenneth C. Litkowski | Suzanne Stevenson
Computational Linguistics, Volume 34, Number 2, June 2008 - Special Issue on Semantic Role Labeling

pdf bib
Simple Semi-supervised Dependency Parsing
Terry Koo | Xavier Carreras | Michael Collins
Proceedings of ACL-08: HLT

2007

pdf bib
Structured Prediction Models via the Matrix-Tree Theorem
Terry Koo | Amir Globerson | Xavier Carreras | Michael Collins
Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL)

pdf bib
Experiments with a Higher-Order Projective Dependency Parser
Xavier Carreras
Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL)

2006

pdf bib
Projective Dependency Parsing with Perceptron
Xavier Carreras | Mihai Surdeanu | Lluís Màrquez
Proceedings of the Tenth Conference on Computational Natural Language Learning (CoNLL-X)

2005

pdf bib
Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling
Xavier Carreras | Lluís Màrquez
Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005)

2004

pdf bib
FreeLing: An Open-Source Suite of Language Analyzers
Xavier Carreras | Isaac Chao | Lluís Padró | Muntsa Padró
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)

pdf bib
Introduction to the CoNLL-2004 Shared Task: Semantic Role Labeling
Xavier Carreras | Lluís Màrquez
Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004

pdf bib
Hierarchical Recognition of Propositional Arguments with Perceptrons
Xavier Carreras | Lluís Màrquez | Grzegorz Chrupała
Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004

2003

pdf bib
A Simple Named Entity Extractor using AdaBoost
Xavier Carreras | Lluís Màrquez | Lluís Padró
Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003

pdf bib
Learning a Perceptron-Based Named Entity Chunker via Online Recognition Feedback
Xavier Carreras | Lluís Màrquez | Lluís Padró
Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003

pdf bib
Low-cost Named Entity Classification for Catalan: Exploiting Multilingual Resources and Unlabeled Data
Lluís Màrquez | Adrià de Gispert | Xavier Carreras | Lluís Padró
Proceedings of the ACL 2003 Workshop on Multilingual and Mixed-language Named Entity Recognition

pdf bib
Named Entity Recognition For Catalan Using Only Spanish Resources and Unlabelled Data
Xavier Carreras | Lluís Màrquez | Lluís Padró
10th Conference of the European Chapter of the Association for Computational Linguistics

2002

pdf bib
Named Entity Extraction using AdaBoost
Xavier Carreras | Lluís Màrquez | Lluís Padró
COLING-02: The 6th Conference on Natural Language Learning 2002 (CoNLL-2002)

pdf bib
A Flexible Distributed Architecture for Natural Language Analyzers
Xavier Carreras | Lluís Padró
Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02)

2001

pdf bib
Boosting trees for clause splitting
Xavier Carreras | Lluís Màrquez
Proceedings of the ACL 2001 Workshop on Computational Natural Language Learning (ConLL)