Yilin Gong
Author directory2026
ShareChatInsights: An Automated Pipeline for Analyzing Large-Scale Human–LLM Conversations
Yueru Yan | Yilin Gong | Siqi Wu | Thai Le
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Yueru Yan | Yilin Gong | Siqi Wu | Thai Le
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Large corpora of conversations between humans and large language models (LLMs) provide a valuable record of how AI systems are used in the wild, but extracting empirical insights from them requires substantial engineering to build custom pipelines that parse multi-platform chat logs and process platform-native artifacts. We present ShareChatInsights, an interactive system that enables natural-language querying and analysis of ShareChat, a large-scale corpus of human-LLM conversations spanning five commercial assistants. An AI agent refines each request through four stages: identify, scope, model, and analyze, turning an open-ended question into a fully specified study that the system executes and exports as a runnable script. Users can configure filters to search, inspect, or export individual conversations on a separate search interface. Users can also opt to publish completed analyses to a shared community gallery. We evaluate the system on 46 requests drawn from prior literature, validate the results against independent LLM and human judgments, and present two empirical case studies. By integrating natural-language querying with reusable analytical infrastructure, ShareChatInsights lowers the barrier to large-scale empirical research on human-LLM conversations for researchers without extensive technical skills.