Daniel Vennemeyer
Author directory2026
Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs
Daniel Vennemeyer | Phan Anh Duong | Tiffany Zhan | Tianyu Jiang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Daniel Vennemeyer | Phan Anh Duong | Tiffany Zhan | Tianyu Jiang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models (LLMs) often exhibit sycophantic behaviors—such as excessive agreement with or flattery of the user—but it is unclear whether these behaviors arise from a single mechanism or multiple distinct processes. We decompose sycophancy into sycophantic agreement and sycophantic praise, contrasting both with genuine agreement. Using difference-in-means directions, activation additions, and subspace geometry across multiple models and datasets, we show that: (1) the three behaviors are encoded along distinct linear directions in latent space; (2) each behavior can be independently amplified or suppressed without affecting the others; and (3) their representational structure is consistent across model families and scales. These results suggest that sycophantic behaviors correspond to distinct, independently steerable representations.
2025
GuessingGame: Measuring the Informativeness of Open-Ended Questions in Large Language Models
Dylan Hutson | Daniel Vennemeyer | Aneesh Deshmukh | Justin Zhan | Tianyu Jiang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Dylan Hutson | Daniel Vennemeyer | Aneesh Deshmukh | Justin Zhan | Tianyu Jiang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
We introduce GuessingGame, a protocol for evaluating large language models (LLMs) as strategic question-askers in open-ended, open-domain settings. A Guesser LLM identifies a hidden object by posing free-form questions to an Oracle—without predefined choices or candidate lists. To measure question quality, we propose two information gain (IG) metrics: a Bayesian method that tracks belief updates over semantic concepts using LLM-scored relevance, and an entropy-based method that filters candidates via ConceptNet. Both metrics are model-agnostic and support post hoc analysis. Across 858 games with multiple models and prompting strategies, higher IG strongly predicts efficiency: a one-standard-deviation IG increase reduces expected game length by 43%. Prompting constraints guided by IG—such as enforcing question diversity—enable weaker models to match GPT-4o. These results show that question-asking in LLMs is both measurable and improvable, and crucial for interactive reasoning.