Syed Mahbubul Huq
Author directory2026
First Make It Playable, Then Make It Good: Staged Interaction Learning for Small Dialogue-Game Agents
Syed Mahbubul Huq | Pranava Madhyastha
The Proceedings of Improving Language Models through Learning from Dialogue Game Interaction
Syed Mahbubul Huq | Pranava Madhyastha
The Proceedings of Improving Language Models through Learning from Dialogue Game Interaction
We present Qwen-GuidePlay-2B, a 2B-parameter language model for dialogue-game interaction. We fine-tune Qwen3.5-2B using three steps: a) SFT on only successful game trajectories from Playpen, b) weighted turn-level SFT, and c) teacher-guided SFT. The teacher model (which is a larger model) is only used to fix formatting and evaluate examples, but does not create new gold actions. Our final model scores 57.12 clemscore and 42.68 statscore on the public Playpen validation. In the officially released challenge results, our model obtains the second-highest Playpen clemscore delta among submitted systems (which is ≈+36 over its base model). Our findings suggest that imitating full trajectories helps with playability, while turn-level and teacher-guided training usually improve decision-making and increase the overall score. Alternative procedurally heavy approaches like replay-repair and hard-example mining did not help, which suggests that small models are performant simply by using careful curation strategies rather than aggressive changes. We make available both the model and the code for reproducibility.
InSight: A Benchmark for Agentic Claim Verification in Interactive Visualizations
Maeve Hutchinson | Syed Mahbubul Huq | Mohammad Albinhassan | Radu Jianu | Aidan Slingsby | Pranava Madhyastha
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Maeve Hutchinson | Syed Mahbubul Huq | Mohammad Albinhassan | Radu Jianu | Aidan Slingsby | Pranava Madhyastha
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Vision Language Models have demonstrated remarkable proficiency in interpreting static visual artifacts, but modern data analysis is inherently dynamic, requiring the active interrogation of interactive environments. Existing benchmarks are predominantly constrained to static imagery and one-shot question answering and fail to capture the epistemic demands of this domain, where evidence is frequently occluded, distributed across linked views, or conditionally revealed through user agency. In this paper, we introduce InSight, a benchmark for agentic claim verification over interactive visualizations. The dataset consists of 21,349 claims derived from human-authored analytical narratives and grounded in fully interactive web-based environments. Agents must navigate these environments to determine whether a natural language claim is supported, refuted or not verifiable given the available evidence. Unlike traditional evaluations, InSight treats interaction traces as intrinsic proxies for reasoning, enabling a rigorous audit of how models seek and synthesize visual evidence. We evaluate state-of-the-art models, revealing that interactive verification remains a non-trivial challenge. We release InSight at https://github.com/maevehutch/insight.