Phong Le


2020

pdf bib
DoLFIn: Distributions over Latent Features for Interpretability
Phong Le | Willem Zuidema
Proceedings of the 28th International Conference on Computational Linguistics

Interpreting the inner workings of neural models is a key step in ensuring the robustness and trustworthiness of the models, but work on neural network interpretability typically faces a trade-off: either the models are too constrained to be very useful, or the solutions found by the models are too complex to interpret. We propose a novel strategy for achieving interpretability that – in our experiments – avoids this trade-off. Our approach builds on the success of using probability as the central quantity, such as for instance within the attention mechanism. In our architecture, DoLFIn (Distributions over Latent Features for Interpretability), we do no determine beforehand what each feature represents, and features go altogether into an unordered set. Each feature has an associated probability ranging from 0 to 1, weighing its importance for further processing. We show that, unlike attention and saliency map approaches, this set-up makes it straight-forward to compute the probability with which an input component supports the decision the neural model makes. To demonstrate the usefulness of the approach, we apply DoLFIn to text classification, and show that DoLFIn not only provides interpretable solutions, but even slightly outperforms the classical CNN and BiLSTM text classifiers on the SST2 and AG-news datasets.

pdf bib
Revisiting Unsupervised Relation Extraction
Thy Thy Tran | Phong Le | Sophia Ananiadou
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Unsupervised relation extraction (URE) extracts relations between named entities from raw text without manually-labelled data and existing knowledge bases (KBs). URE methods can be categorised into generative and discriminative approaches, which rely either on hand-crafted features or surface form. However, we demonstrate that by using only named entities to induce relation types, we can outperform existing methods on two popular datasets. We conduct a comparison and evaluation of our findings with other URE techniques, to ascertain the important features in URE. We conclude that entity types provide a strong inductive bias for URE.

2019

pdf bib
Boosting Entity Linking Performance by Leveraging Unlabeled Documents
Phong Le | Ivan Titov
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Modern entity linking systems rely on large collections of documents specifically annotated for the task (e.g., AIDA CoNLL). In contrast, we propose an approach which exploits only naturally occurring information: unlabeled documents and Wikipedia. Our approach consists of two stages. First, we construct a high recall list of candidate entities for each mention in an unlabeled document. Second, we use the candidate lists as weak supervision to constrain our document-level entity linking model. The model treats entities as latent variables and, when estimated on a collection of unlabelled texts, learns to choose entities relying both on local context of each mention and on coherence with other entities in the document. The resulting approach rivals fully-supervised state-of-the-art systems on standard test sets. It also approaches their performance in the very challenging setting: when tested on a test set sampled from the data used to estimate the supervised systems. By comparing to Wikipedia-only training of our model, we demonstrate that modeling unlabeled documents is beneficial.

pdf bib
Distant Learning for Entity Linking with Automatic Noise Detection
Phong Le | Ivan Titov
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Accurate entity linkers have been produced for domains and languages where annotated data (i.e., texts linked to a knowledge base) is available. However, little progress has been made for the settings where no or very limited amounts of labeled data are present (e.g., legal or most scientific domains). In this work, we show how we can learn to link mentions without having any labeled examples, only a knowledge base and a collection of unannotated texts from the corresponding domain. In order to achieve this, we frame the task as a multi-instance learning problem and rely on surface matching to create initial noisy labels. As the learning signal is weak and our surrogate labels are noisy, we introduce a noise detection component in our model: it lets the model detect and disregard examples which are likely to be noisy. Our method, jointly learning to detect noise and link entities, greatly outperforms the surface matching baseline. For a subset of entity categories, it even approaches the performance of supervised learning.

2018

pdf bib
Improving Entity Linking by Modeling Latent Relations between Mentions
Phong Le | Ivan Titov
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Entity linking involves aligning textual mentions of named entities to their corresponding entries in a knowledge base. Entity linking systems often exploit relations between textual mentions in a document (e.g., coreference) to decide if the linking decisions are compatible. Unlike previous approaches, which relied on supervised systems or heuristics to predict these relations, we treat relations as latent variables in our neural entity-linking model. We induce the relations without any supervision while optimizing the entity-linking system in an end-to-end fashion. Our multi-relational model achieves the best reported scores on the standard benchmark (AIDA-CoNLL) and substantially outperforms its relation-agnostic version. Its training also converges much faster, suggesting that the injected structural bias helps to explain regularities in the training data.

2017

pdf bib
Optimizing Differentiable Relaxations of Coreference Evaluation Metrics
Phong Le | Ivan Titov
Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)

Coreference evaluation metrics are hard to optimize directly as they are non-differentiable functions, not easily decomposable into elementary decisions. Consequently, most approaches optimize objectives only indirectly related to the end goal, resulting in suboptimal performance. Instead, we propose a differentiable relaxation that lends itself to gradient-based optimisation, thus bypassing the need for reinforcement learning or heuristic modification of cross-entropy. We show that by modifying the training objective of a competitive neural coreference system, we obtain a substantial gain in performance. This suggests that our approach can be regarded as a viable alternative to using reinforcement learning or more computationally expensive imitation learning.

2016

pdf bib
Quantifying the Vanishing Gradient and Long Distance Dependency Problem in Recursive Neural Networks and Recursive LSTMs
Phong Le | Willem Zuidema
Proceedings of the 1st Workshop on Representation Learning for NLP

pdf bib
LSTM-Based Mixture-of-Experts for Knowledge-Aware Dialogues
Phong Le | Marc Dymetman | Jean-Michel Renders
Proceedings of the 1st Workshop on Representation Learning for NLP

2015

pdf bib
The Forest Convolutional Network: Compositional Distributional Semantics with a Neural Chart and without Binarization
Phong Le | Willem Zuidema
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

pdf bib
Enhancing the Inside-Outside Recursive Neural Network Reranker for Dependency Parsing
Phong Le
Proceedings of the 14th International Conference on Parsing Technologies

pdf bib
Unsupervised Dependency Parsing: Let’s Use Supervised Parsers
Phong Le | Willem Zuidema
Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

pdf bib
Compositional Distributional Semantics with Long Short Term Memory
Phong Le | Willem Zuidema
Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics

2014

pdf bib
The Inside-Outside Recursive Neural Network model for Dependency Parsing
Phong Le | Willem Zuidema
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)

2013

pdf bib
Learning from errors: Using vector-based compositional semantics for parse reranking
Phong Le | Willem Zuidema | Remko Scha
Proceedings of the Workshop on Continuous Vector Space Models and their Compositionality

2012

pdf bib
Learning Compositional Semantics for Open Domain Semantic Parsing
Phong Le | Willem Zuidema
Proceedings of COLING 2012