Deepak Ramachandran


2023

pdf bib
KwikBucks: Correlation Clustering with Cheap-Weak and Expensive-Strong Signals
Sandeep Silwal | Sara Ahmadian | Andrew Nystrom | Andrew Mccallum | Deepak Ramachandran | Mehran Kazemi
Proceedings of The Fourth Workshop on Simple and Efficient Natural Language Processing (SustaiNLP)

pdf bib
LAMBADA: Backward Chaining for Automated Reasoning in Natural Language
Mehran Kazemi | Najoung Kim | Deepti Bhatia | Xin Xu | Deepak Ramachandran
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Remarkable progress has been made on automated reasoning with natural text, by using Large Language Models (LLMs) and methods such as Chain-of-Thought prompting and Selection-Inference. These techniques search for proofs in the forward direction from axioms to the conclusion, which suffers from a combinatorial explosion of the search space, and thus high failure rates for problems requiring longer chains of reasoning. The classical automated reasoning literature has shown that reasoning in the backward direction (i.e. from intended conclusion to supporting axioms) is significantly more efficient at proof-finding. Importing this intuition into the LM setting, we develop a Backward Chaining algorithm, called LAMBADA, that decomposes reasoning into four sub-modules, that are simply implemented by few-shot prompted LLM inference. We show that LAMBADA achieves sizable accuracy boosts over state-of-the-art forward reasoning methods on two challenging logical reasoning datasets, particularly when deep and accurate proof chains are required.

pdf bib
Using Domain Knowledge to Guide Dialog Structure Induction via Neural Probabilistic Soft Logic
Connor Pryor | Quan Yuan | Jeremiah Liu | Mehran Kazemi | Deepak Ramachandran | Tania Bedrax-Weiss | Lise Getoor
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Dialog Structure Induction (DSI) is the task of inferring the latent dialog structure (i.e., a set of dialog states and their temporal transitions) of a given goal-oriented dialog. It is a critical component for modern dialog system design and discourse analysis. Existing DSI approaches are often purely data-driven, deploy models that infer latent states without access to domain knowledge, underperform when the training corpus is limited/noisy, or have difficulty when test dialogs exhibit distributional shifts from the training domain. This work explores a neural-symbolic approach as a potential solution to these problems. We introduce Neural Probabilistic Soft Logic Dialogue Structure Induction (NEUPSL DSI), a principled approach that injects symbolic knowledge into the latent space of a generative neural model. We conduct a thorough empirical investigation on the effect of NEUPSL DSI learning on hidden representation quality, few-shot learning, and out-of-domain generalization performance. Over three dialog structure induction datasets and across unsupervised and semi-supervised settings for standard and cross-domain generalization, the injection of symbolic knowledge using NEUPSL DSI provides a consistent boost in performance over the canonical baselines.

2022

pdf bib
FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue
Alon Albalak | Yi-Lin Tuan | Pegah Jandaghi | Connor Pryor | Luke Yoffe | Deepak Ramachandran | Lise Getoor | Jay Pujara | William Yang Wang
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing

Task transfer, transferring knowledge contained in related tasks, holds the promise of reducing the quantity of labeled data required to fine-tune language models. Dialogue understanding encompasses many diverse tasks, yet task transfer has not been thoroughly studied in conversational AI. This work explores conversational task transfer by introducing FETA: a benchmark for FEw-sample TAsk transfer in open-domain dialogue.FETA contains two underlying sets of conversations upon which there are 10 and 7 tasks annotated, enabling the study of intra-dataset task transfer; task transfer without domain adaptation. We utilize three popular language models and three learning algorithms to analyze the transferability between 132 source-target task pairs and create a baseline for future work. We run experiments in the single- and multi-source settings and report valuable findings, e.g., most performance trends are model-specific, and span extraction and multiple-choice tasks benefit the most from task transfer. In addition to task transfer, FETA can be a valuable resource for future research into the efficiency and generalizability of pre-training datasets and model architectures, as well as for learning settings such as continual and multitask learning.

2021

pdf bib
Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering
Najoung Kim | Ellie Pavlick | Burcu Karagol Ayan | Deepak Ramachandran
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)

Many Question-Answering (QA) datasets contain unanswerable questions, but their treatment in QA systems remains primitive. Our analysis of the Natural Questions (Kwiatkowski et al. 2019) dataset reveals that a substantial portion of unanswerable questions (~21%) can be explained based on the presence of unverifiable presuppositions. Through a user preference study, we demonstrate that the oracle behavior of our proposed system—which provides responses based on presupposition failure—is preferred over the oracle behavior of existing QA systems. Then, we present a novel framework for implementing such a system in three steps: presupposition generation, presupposition verification, and explanation generation, reporting progress on each. Finally, we show that a simple modification of adding presuppositions and their verifiability to the input of a competitive end-to-end QA system yields modest gains in QA performance and unanswerability detection, demonstrating the promise of our approach.

2020

pdf bib
Do Language Embeddings capture Scales?
Xikun Zhang | Deepak Ramachandran | Ian Tenney | Yanai Elazar | Dan Roth
Findings of the Association for Computational Linguistics: EMNLP 2020

Pretrained Language Models (LMs) have been shown to possess significant linguistic, common sense and factual knowledge. One form of knowledge that has not been studied yet in this context is information about the scalar magnitudes of objects. We show that pretrained language models capture a significant amount of this information but are short of the capability required for general common-sense reasoning. We identify contextual information in pre-training and numeracy as two key factors affecting their performance, and show that a simple method of canonicalizing numbers can have a significant effect on the results.

pdf bib
Do Language Embeddings capture Scales?
Xikun Zhang | Deepak Ramachandran | Ian Tenney | Yanai Elazar | Dan Roth
Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP

Pretrained Language Models (LMs) have been shown to possess significant linguistic, common sense and factual knowledge. One form of knowledge that has not been studied yet in this context is information about the scalar magnitudes of objects. We show that pretrained language models capture a significant amount of this information but are short of the capability required for general common-sense reasoning. We identify contextual information in pre-training and numeracy as two key factors affecting their performance, and show that a simple method of canonicalizing numbers can have a significant effect on the results.

2019

pdf bib
How Large Are Lions? Inducing Distributions over Quantitative Attributes
Yanai Elazar | Abhijit Mahabal | Deepak Ramachandran | Tania Bedrax-Weiss | Dan Roth
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Most current NLP systems have little knowledge about quantitative attributes of objects and events. We propose an unsupervised method for collecting quantitative information from large amounts of web data, and use it to create a new, very large resource consisting of distributions over physical quantities associated with objects, adjectives, and verbs which we call Distributions over Quantitative (DoQ). This contrasts with recent work in this area which has focused on making only relative comparisons such as “Is a lion bigger than a wolf?”. Our evaluation shows that DoQ compares favorably with state of the art results on existing datasets for relative comparisons of nouns and adjectives, and on a new dataset we introduce.

2015

pdf bib
Belief Tracking with Stacked Relational Trees
Deepak Ramachandran | Adwait Ratnaparkhi
Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue

pdf bib
A TV Program Discovery Dialog System using recommendations
Deepak Ramachandran | Mark Fanty | Ronald Provine | Peter Yeh | William Jarrold | Adwait Ratnaparkhi | Benjamin Douglas
Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue

2013

pdf bib
The Dialog State Tracking Challenge
Jason Williams | Antoine Raux | Deepak Ramachandran | Alan Black
Proceedings of the SIGDIAL 2013 Conference

2012

pdf bib
Landmark-Based Location Belief Tracking in a Spoken Dialog System
Yi Ma | Antoine Raux | Deepak Ramachandran | Rakesh Gupta
Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue

2010

pdf bib
Probabilistic Ontology Trees for Belief Tracking in Dialog Systems
Neville Mehta | Rakesh Gupta | Antoine Raux | Deepak Ramachandran | Stefan Krawczyk
Proceedings of the SIGDIAL 2010 Conference