Snehanshu Saha

Author directory

2026

Existing LLM data-analyst benchmarks fix data access at full disclosure and evaluate only on answer accuracy. But real deployments rarely grant full access. No prior work, to our knowledge, asks *"what’s the least amount of information needed for the agent to work?"*. In this article, we treat dataset disclosure as a controlled treatment along a *graduated information-disclosure* (GID) axis: C0 (column names) and C∞ (full table) bracket a nested chain C1 ⊆ … ⊆ C5 that adds per-column metadata, dataset purpose, summary statistics, sampled cells, and sampled rows. We maintain agent architecture, tool registry, and decoding fixed, and score the final reports on a 14-attribute rubric across a 6 × 4 × 12 panel of frontier models, planted-signal datasets, and disclosure levels. At a high level, from C0 to C∞, seven attributes rise and five fall. However, the rise/fall pattern has nuance, the optimal intermediate level depends on several factors including models, datasets and attributes. Level C1 - one sentence of types and units beyond column names - matches (statistically) C∞ on signal recovery and decoy rejection at ∼49× lower prompt-token cost. The key takeaway from this work is - **current LLM data-science agents can act on partial information alone**, so row-level records need never leave the institution. Hence, the current framework also offers a way to enforce data boundary constraints (required by GDPR) without compromising on quality.

2018

In this work we present our participation to SemEval-2018 Task 8 subtasks 1 & 2 respectively. We developed Convolution Neural Network system for malware sentence classification (subtask 1) and Conditional Random Fields system for malware token label prediction (subtask 2). We experimented with couple of word embedding strategies, feature sets and achieved competitive performance across the two subtasks. For subtask 1 We experimented with two category of word embeddings namely native embeddings and task specific embedding using Word2vec and Glove algorithms. 1. Native Embeddings: All words including the unknown ones that are randomly initialized use embeddings from original Word2vec/Glove models. 2. Task specific : The embeddings are generated by training Word2vec/Glove algorithms on sentences from MalwareTextDB We found that glove outperforms rest of embeddings for subtask 1. For subtask 2, we used N-grams of size 6, previous, next tokens and labels, features giving disjunctions of words anywhere in the left or right, word shape features, word lemma of current, previous and next words, word-tag pair features, POS tags, prefix and suffixes.