QUIS: Question-guided Insights Generation for Automated Exploratory Data Analysis

Abhijit Manatkar, Ashlesha Akella, Parthivi Gupta, Krishnasuri Narayanam


Abstract
Discovering meaningful insights from a large dataset, known as Exploratory Data Analysis (EDA), is a challenging task that requires thorough exploration and analysis of the data. Automated Data Exploration (ADE) systems use goal-oriented methods with Large Language Models and Reinforcement Learning towards full automation. However, these methods require human involvement to anticipate goals that may limit insight extraction, while fully automated systems demand significant computational resources and retraining for new datasets. We introduce QUIS, a fully automated EDA system that operates in two stages: insight generation (ISGen) driven by question generation (QUGen). The QUGen module generates questions in iterations, refining them from previous iterations to enhance coverage without human intervention or manually curated examples. The ISGen module analyzes data to produce multiple relevant insights in response to each question, requiring no prior training and enabling QUIS to adapt to new datasets.
Anthology ID:
2024.emnlp-industry.111
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
November
Year:
2024
Address:
Miami, Florida, US
Editors:
Franck Dernoncourt, Daniel Preoţiuc-Pietro, Anastasia Shimorina
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1523–1535
Language:
URL:
https://aclanthology.org/2024.emnlp-industry.111
DOI:
Bibkey:
Cite (ACL):
Abhijit Manatkar, Ashlesha Akella, Parthivi Gupta, and Krishnasuri Narayanam. 2024. QUIS: Question-guided Insights Generation for Automated Exploratory Data Analysis. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1523–1535, Miami, Florida, US. Association for Computational Linguistics.
Cite (Informal):
QUIS: Question-guided Insights Generation for Automated Exploratory Data Analysis (Manatkar et al., EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-industry.111.pdf
Presentation:
 2024.emnlp-industry.111.presentation.pdf