Large language model (LLM)-based multi-agent systems (MAS) have shown strong potential for solving complex tasks, yet their development forces a tradeoff: code frameworks are expressive but engineering-intensive, while no-code builders simplify authoring but constrain agent interactions to author-defined workflows. We present **ChatDev 2.0: DevAll** (hereafter DevAll), a no-code platform for building, executing, and inspecting heterogeneous MAS that delivers both high expressiveness and ease of use. In terms of expressiveness, DevAll pairs a declarative executable graph abstraction with a cycle-aware execution engine, so that heterogeneous agents and dynamic and cyclic interactions can be represented and executed within a single framework. For ease of use, an integrated visual interface lets users author, run, monitor, and inspect MAS, including human-in-the-loop steps, entirely without writing code. Experiments demonstrate that DevAll reproduces state-of-the-art MAS across three representative tasks at competitive performance and without task-specific orchestration code, highlighting its effectiveness as a general-purpose platform for LLM-based MAS. DevAll is available at https://github.com/OpenBMB/ChatDev.

Large Language Models (LLMs) increasingly provide direct answers to user questions, raising concerns about reduced engagement in critical thinking and problem-solving. Hint generation offers an alternative by guiding users toward answers without revealing them, while hint evaluation assesses the quality of such guidance. Research in this area is hindered by fragmented datasets, inconsistent annotation formats, and evaluation tools that are often dataset-specific or unavailable. To address these challenges, we introduce HintEval, an open-source Python library for unified hint generation and evaluation. HintEval standardizes access to diverse hint datasets, supports answer-aware and answer-agnostic generation methods, and implements multiple evaluation metrics within a shared data model. The toolkit enables reproducible experimentation, cross-dataset analysis, and multi-dimensional evaluation with minimal engineering effort. We further demonstrate its utility through human studies in which participants assess generated hints and use them to answer questions, showing that hints can effectively support users in reaching correct answers. HintEval is accompanied by comprehensive documentation, an executable Google Colab notebook for rapid experimentation, and a demonstration video. By promoting consistent evaluation practices and lowering barriers to entry, it facilitates systematic research on hint-based question answering (QA) in NLP and IR.

Large language model (LLM) steering has emerged as a promising paradigm for controlling model behavior at inference time through targeted manipulation of hidden states, offering a lightweight alternative to expensive retraining. However, existing steering frameworks suffer from critical limitations: computational inefficiency, limited extensibility, and restricted functionality that hinder both research progress and practical deployment. We present EasySteer, a unified framework for high-performance, extensible LLM steering built on vLLM. Our system features modular architecture with pluggable interfaces for both analysis-based and learning-based methods, fine-grained parameter control, pre-computed steering vectors for eight application domains, and an interactive demonstration system. Through deep integration with vLLM’s optimized inference engine, EasySteer achieves 10.8-22.3× speedup over existing frameworks. Extensive experiments demonstrate its effectiveness in overthinking mitigation, hallucination reduction, and other key applications. EasySteer transforms steering from research technique to production-ready capability, establishing critical infrastructure for deployable, controllable language models.

The emergence of Large Reasoning Models has introduced exceptionally long Chain-of-Thought traces, creating a transparency burden where critical logic is often buried under massive procedural text. To address this, we present ReasoningLens, an open-source framework designed for the hierarchical visualization and diagnostic auditing of complex reasoning chains. ReasoningLens addresses information necropsy by: (1) structuring traces into interactive hierarchies that separate high-level strategy from low-level execution; (2) leveraging an agentic auditor for automated error detection and tool-augmented verification; and (3) synthesizing systemic reasoning profiles to reveal model-specific blind spots. By transforming unstructured walls of text into actionable insights, ReasoningLens provides a modular foundation for interpreting, debugging, and optimizing the next generation of reasoning-centric AI.

Classical Chinese poetry composition has long valued *Tuiqiao*, the iterative refinement of words, imagery, and prosody. However, many current AI poetry systems follow a one-shot generation paradigm, which reduces users to prompt providers and weakens their creative agency. We present *Jiuge-Tuiqiao*, an interactive human-AI collaborative system for classical Chinese poetry composition. The system is designed around a triadic model: user-driven control, ancient-guided evidence, and AI-assisted generation. Users can lock characters or lines, receive real-time prosody feedback, and obtain interpretable refinement suggestions grounded in high-frequency collocations, PPL-ranked classical lines, and structured knowledge extracted from classical encyclopedias. This design turns AI from an autonomous generator into a background assistant that supports the user’s own process of poetic refinement. Preliminary experiments and user feedback suggest that *Jiuge-Tuiqiao* improves controllability, interpretability, and user engagement in classical poetry composition.

Good tutoring adapts to the individual: it tracks what a learner knows, notices why they go wrong, and asks the next question that will help most. Most deployed tutoring tools instead serve fixed item banks and treat a wrong answer as a single bit of signal. We present CoLearn, an interactive, agentic tutor that supports an iterative tutoring loop: the learner practises, and the system builds an evidence-grounded memory of the learner’s mastery and misconceptions. This memory is updated as evidence accumulates and is used to generate the next personalised question. CoLearn has three components: (i) a persistent learner-state memory that updates per-topic mastery with a soft-evidence variant of Bayesian Knowledge Tracing, where a large language model acts as a continuous ob- servation function; (ii) adaptive question generation that targets the learner’s weakest topic and recurring misconceptions; and (iii) an evi- dence view that makes personalisation visible and testable through live progress visualisation and blind A/B comparison. In blind A/B evaluation, questions conditioned on this memory are preferred over non-personalised ones 68–69% of the time, and in persona simulations with hidden ground-truth mastery the agent’s belief converges toward the learner’s true mastery.

Human evaluation is the gold standard for multilingual NLP, but is often skipped in practice and substituted with automatic metrics, because it is notoriously complex and slow to set up with existing tools with substantial engineering and operational overhead. We introduce Pearmut, a lightweight yet feature-rich platform that makes end-to-end human evaluation as easy to run as automatic evaluation. Pearmut removes common entry barriers and provides support for evaluating multilingual tasks, with a particular focus on machine translation. The platform implements standard evaluation protocols, including DA, ESA, or MQM, but is also extensible to allow prototyping new protocols. It features document-level context, absolute and contrastive evaluation, attention checks, ESAAI pre-annotations and both static and active learning-based assignment strategies. Pearmut enables reliable human evaluation to become a practical, routine component of model development and diagnosis rather than an occasional effort.

Streaming Speech-to-Text Translation (StreamST) requires producing translations concurrently with incoming speech under strict latency constraints, demanding models that balance low latency with high translation quality. Despite rapid progress, evaluation remains fragmented across existing frameworks, which make different assumptions about how systems operate—for example, whether they process continuous speech or short pre-segmented audio, and whether they support output revision (retranslation) or not (incremental) during decoding. As a result, comparing systems fairly and consistently across studies remains challenging. SimulEval, the most widely used framework, reflects these limitations: it supports only incremental decoding, assumes short segmented inputs, and lacks a native support for system demonstrations. More broadly, existing alternatives address only subsets of evaluation and deployment needs, leaving no unified solution for benchmarking and interactive demonstration. To address this gap, we introduce simulstream, the first open-source framework for StreamST evaluation and demonstration. It supports both incremental and re-translation decoding on long-form speech, provides fine-grained logging for quality and latency evaluation, and includes an interactive web interface for real-time visualization and comparison.

Traditional approaches to wearable health signal analysis, such as smartwatches, are constrained by rigid analytical frameworks and limited personalisation. The emergence of LLM agents creates a new opportunity for Personal Health Agentic Analysis, where health insights can be generated adaptively and in context. However, currently there is no open-source locally deployable platform capable of processing personal health data in real time while preserving privacy. We present , a locally deployable, privacy-first agent platform that is fully compatible with real-time health data ecosystems across a wide range of wearable devices. is guided by three design principles. The database is treated as a first-class component. Effectiveness and efficiency are jointly optimised to achieve a low-cost Pareto-optimal balance. Data are processed in real time while the user is modelled over the long term. Together, these principles make it practical for individuals to harness Personal Health Agents for continuous, personalised health monitoring for better wellbeing.

Regulatory compliance checking-deciding whether a target document satisfies the obligations of a regulation-requires interpreting dense legal text, identifying which provisions apply, and grounding each decision in explicit evidence. We present ARCCS, an end-to-end, automated, agentic, and regulation-agnostic Legal NLP system for compliance checking. ARCCS decomposes raw regulatory text into atomic, traceable requirements and evaluates a target document against them using retrieved evidence, confidence scores, and human-interpretable justifications. This design decouples compliance assessment from any fixed regulatory template or predefined rule set, enabling the pipeline to operate over regulations of varying size and structure. We evaluate ARCCS in two complementary settings. First, in a GDPR policy-document evaluation, LLM-based judges find its decisions and justifications legally and evidentially consistent in up to 96.67% of the assessed cases. Second, on an EU public-procurement benchmark comprising more than 1,200 individual rule checks, the system attains 98.8% accuracy in violation detection. ARCCS is, to our knowledge, the first fully open-source system for end-to-end regulatory compliance checking and auditable report generation.

Fairness metrics such as demographic parity and equalized odds summarize group-level behavior, while the location of subgroup errors in the score distribution often remains unclear. RISE-NLP is a diagnostic interface for auditing probabilistic text classifiers through signed residual distributions. Given prediction scores, labels, group metadata, and text, RISE-NLP plots group residual-quantile curves, reports distributional metrics, and links selected percentile regions to the examples behind the gap. Using BERT and RoBERTa predictions, we observe F_dist=0.097--0.150 in the two identity-linked toxicity settings and 0.055--0.058 in a randomly grouped SST-2 control used as a descriptive reference. A 16-participant formative study evaluates users’ understanding and perceived usefulness of the residual visualizations, showing their potential to support reasoning about fairness–accuracy trade-offs. The project is available at https://ufdatastudio.com/projects/risenlp.

Manga pages combine text, layout, characters, and narrative order, making them difficult to process as plain images or text alone. We present MangaLens, an integrated tool for structured manga parsing. MangaLens combines LLM-based reasoning with segmentation-based grounding to support manga parsing subtasks such as panel structuring, speaker linking, and related cross-page reasoning tasks. The system produces page-level and book-level JSON records and visualizes subtask outputs with aligned image evidence, enabling users to verify, correct, and reuse parsing artifacts. Evaluation results show that MangaLens produces reliable structured parsing outputs across major subtasks. The tool is released at https://github.com/nict-astrec-att/MangaLens

Scientific abstracts mix contributions with background, motivation, and meta-language, so tools that read them as-is cannot separate what a field produces from what it discusses. We present Drift Inspector, an open-source system for measuring and exploring how a research field changes over time at the level of Atomic Contribution Claims (ACCs): decontextualized, contribution-bearing propositions an LLM extracts from each abstract before analysis. The system clusters these claims across years into an interactive map where every trend traces back to the claims and papers behind it. Applied to six years of EMNLP, it shows the field shifting away from classic NLP tasks toward LLM-era capabilities such as reasoning and multimodality — a movement that keyword or whole-abstract counts blur. The released data extend beyond EMNLP: the same pipeline has processed the full ACL Anthology (346k claims, 80k abstracts, 423 venues). Extraction is human-validated and clustering checked against an external manually constructed taxonomy.

Online debates on platforms like Reddit or Kialo are structured as reply trees: each message can receive several replies, which in turn can each receive several replies, producing a branching structure rather than a linear conversation. Turning one path (or a mix of paths) through such a tree into a natural, turn-by-turn conversation among multiple speakers (a process we call linearization) requires deciding which messages to keep, in what order, and who each message is addressed to. To this aim, we present LLMberjack, a platform for creating multi-party conversations starting from existing debates, originally structured as reply trees. The system offers an interactive interface that visualizes discussion trees and enables users to construct coherent linearized dialogue sequences while preserving participant identity and discourse relations. It integrates optional large language model (LLM) assistance to support automatic editing of the messages and speakers’ descriptions. We provide preliminary evidence about the platform’s utility by showing how tree visualization facilitates the creation of coherent, meaningful conversation threads and how LLM support enhances output quality while reducing human effort. The tool is open-source and designed to promote transparent and reproducible workflows to create multi-party conversations, addressing a lack of resources of this type.

Autonomous GUI and web agents frequently generate long, multi-step execution traces with rich environmental states, making manual inspection and fault localization challenging. Existing visualization tools typically render isolated single-run trajectories, obscuring fine-grained behavioral discrepancies across parallel runs on the same task. We introduce TRACEFORK, an interactive system designed to align, compare, and inspect paired same-task trajectories produced by AI agents or human demonstrators. TRACEFORK ingests execution traces from standardized benchmarks (e.g., WebArena, VisualWebArena) and custom user logs via an extensible normalization schema. Using a multi-modal dynamic programming alignment engine, the system collapses shared prefixes onto a single trunk, isolates behavioral divergence points into distinct branches, and identifies downstream reconvergence via conservative rejoin cues. Across a library of 1,722 tasks and 1,387 aligned trajectory pairs (33,337 steps), TRACEFORK achieves a median alignment latency of 45.5 ms per pair (p95 320.4 ms). On a manually annotated pilot subset, it achieves an F1 score of 0.910 for divergence localization, offering an open-source, deterministic diagnostic workbench for the agent research community.

Personalized AI assistants in collaborative settings must balance private user context with shared team knowledge; i.e. the same question can have different answers, depending on who is asking. However, retrieving and reasoning over a growing shared archive, as agents do today, increases latency and cost. We present Goldilocks, a multimodal assistive-agent demo that explores a tiered personal memory architecture which combines a lightweight private working resident memory alongside a shared team memory, both grounded in a common eventstore. A sufficiency gate first tries to answer from the low-cost resident tier and only escalates other tiers when needed. To evaluate long-lived personalized collaborative assistants, we also introduce WhoQA, a 500-question evaluation set built from public GitHub history, where work-related questions and answers trace to real pull requests and commits. On WhoQA, Goldilocks’s deployed small-model configuration beats a strong hybrid RAG baseline by 7.0 points (59.0 vs. 52.0) at 13% lower estimated energy (a tier-weighted token cost), and reaches 62.4 (+10.4) with the answer model held fixed. The system serves 42.6% of queries from the resident tier. A compact social-memory scoping further lifts accuracy on teammate-specific personal details from 53 to 80. While we demonstrate with a set tier and scoping configuration, the system is generalizable to multiple tiers depending on the use case. The live demo lets users explore memory tiers, gating decisions, and interactively compare personalized responses across team personas.

Safety evaluation of multimodal large language models requires tracking not only whether an attack succeeds, but also how the interaction unfolds across turns and input modalities. We present MUSE (Multimodal Unified Safety Evaluation), an open-source, browser-based, run-centric platform for multimodal safety evaluation. MUSE treats each attack run as the persistent unit of execution, inspection, and analysis, preserving its configuration, multi-turn trajectory, delivered modalities and media, target responses, and safety judgments. A five-level response taxonomy further distinguishes full Compliance from Partial Compliance and refusal behavior, yielding hard ASR, soft ASR, and gray-zone width (GZW).

Across 11,700 evaluations on six multimodal LLMs, direct text-only requests yield only 3.1% macro hard ASR and 4.4% soft ASR, while iterative attack procedures are substantially more effective. Attack effectiveness also varies substantially with the attacker backbone. In contrast, Inter-Turn Modality Switching (ITMS), evaluated as a controlled delivery-modality probe, does not consistently increase attack success. These results demonstrate the value of run-centric, fine-grained evaluation for characterizing multimodal safety behavior beyond a single binary success metric.

We present ‘gradiend‘, an open-source Python package that operationalizes the GRADIEND method for learning feature directions from factual-counterfactual MLM and CLM gradients in language models. The package provides a unified workflow for feature-related data creation, training, evaluation, visualization, persistent model rewriting via controlled weight updates, and multi-feature comparison. We demonstrate ‘gradiend‘ through an English pronoun running example, a semantic sentiment use case that evaluates lexical generalization to held-out target words, and a large-scale feature comparison.

Email is becoming the universal interface for AI agents, but an ingested message contains un- trusted, malicious data the agent may execute as instructions. The indirect prompt-injection failure underlies real-world incidents such as EchoLeak (M365 Copilot, CVE-2025-32711) and the Gemini-in-Gmail hidden-HTML hijack. We present e2a, a deployable, open-source authenticated-email gateway that gives an agent a verified email identity and exposes two agent- agnostic defense hook points. The (G)ateway authenticates senders via SPF/DKIM/DMARC and parses MIME structure; the (L)LM-layer screens the normalized prompt for injection. On top of the gateway, we construct a bench- mark that isolates email structure as the first- class injection surface: one payload rendered across eight transport surfaces. We measure the per-surface evasion delta for open-source classifiers, three commercial security APIs, and frontier LLM judges. We find that structural hiding defeats detection. A naive text/plain- first parse costs every detector substantial AUC, dropping most classifiers below chance, and even on the gateway’s structure-aware view no non-LLM detector survives. A frontier LLM judge on the same view recovers most of the loss, and conditioning the threshold on crypto- graphic sender authentication (DMARC) lifts held-out recall further. No single layer suf- fices. e2a is live and open-source (Apache-2.0) at https://e2a.dev, released with the bench- mark and harness.

The Model Context Protocol (MCP) lets large language model agents call external tools, and that tool channel has become a security liability: poisoned tool metadata, injected instructions in tool responses, and malicious servers can subvert an agent while never touching the user’s prompt. These risks are increasingly well documented, but hard to experience: most defenses exist as prose or prototypes. We present SHIELDMCP, an interactive, deployment-ready system that makes both the attack and the runtime defense tangible in a browser. SHIELDMCP operates as a transparent proxy between the LLM client and any MCP server, performing threestage validation: pre-call description integrity, outbound parameter sanitization, and postresponse analysis, all without modifying either the agent or the server. The demonstration exposes the full pipeline through a browser interface in which visitors launch curated attacks, watch each validation stage execute live with per-token highlighting of the detected payload, and submit their own tool descriptions, call parameters, or tool responses to a playground that runs the real detector. On a framework benchmark of 487 attack and 200 benign scenarios across 40 MCP servers, the default rule-based backend reduces the overall attack success rate to 8.6% (from a 70.9% undefended baseline measured against live agents in a companion study) at a median added latency under 20 ms; an optional fine-tuned classifier backend reaches near-zero on this benchmark, though we show this is a withindistribution result that does not yet transfer to paraphrased, multilingual, or encoded attacks. SHIELDMCP is open-source under the Apache-2.0 license and installable with a single pip command; every framework number in this paper is reproducible from the shipped harness.

Large language models (LLMs) are increasingly evolving from conversational assistants into agents capable of operating external digital environments. Graphical user interface (GUI) agents are an important part of this shift, as many real-world workflows remain accessible only through user-facing software interfaces. However, despite recent progress on general computer-use benchmarks, domain-specific professional standard operating procedures (SOPs) remain challenging for GUI agents because they often involve implicit domain knowledge, software-specific conventions, and task-level verification requirements. We introduce OmegaUse-SOP, a human-in-the-loop SOP engineering system for transforming human demonstrations of professional computer use into reusable SOP skills for GUI agents. Analogous to prompt engineering, SOP Engineering iteratively refines demonstrations, execution rules, and domain knowledge to convert professional SOPs into reusable GUI-agent skills. OmegaUse-SOP consists of four modules: Observe, Reason, Configure, and Execute. Together, these modules record expert operations as multimodal GUI traces, abstract low-level events into semantic step-level instructions, incorporate domain rules and task-specific parameters, and execute the resulting skills in live GUI environments through step-wise grounding, action generation, and verification. To demonstrate its effectiveness, we collaborate with a power-sector client and test OmegaUse-SOP on photovoltaic simulation workflows in PVsyst 7.2. The results suggest that OmegaUse-SOP can improve GUI-agent reliability on professional SOP tasks, highlighting a practical path toward deploying GUI agents in domain-specific professional software environments.

Agentic coding tools present new opportunities to transform research workflows. The performance of agent systems built depends on both large language models (LLMs) and the harness around LLMs, which is the orchestration code that determines an agent’s behavior. We present ToFu, an agentic harness for researchers that reads your codebase, edits files, runs commands, and integrates with your development tools.

ToFu plays a dual role in research. As a research assistant, it supports practical research workflows with superior token efficiency, lower cost, and multilingual capability compared with existing agentic harnesses. Its release under the MIT License further enables local deployment for privacy-sensitive users. As a research object, ToFu provides a white-box agentic harness that allows researchers to inspect, modify, and evaluate its orchestration logic, tool-use behavior, and harness design, while retaining strong benchmark performance and an application-level user experience.

Ancient manuscripts are the primary source of ancient linguistic corpora. However, some of them exhibit duplications due to unintentional repeated publication or deliberate forgery. Notable examples include counterfeit fragments within the Dead Sea Scrolls and the repeated publication of Oracle Bones (OB). Identifying such duplicates is critical for both archaeological curation and historical research. We present OBD-Finder, an explainable coarse-to-fine framework for large-scale duplicate discovery in ancient manuscript collections, with a primary focus on Oracle Bones. OBD-Finder integrates unsupervised low-level keypoint matching with high-level character-centric content similarity to progressively filter and rank duplicate candidates, balancing retrieval accuracy with computational efficiency. Designed as a practical, transparent, and domain-expert-friendly tool, the system produces candidate duplicates and supports expert inspection and validation. In real-world deployment, OBD-Finder has enabled domain experts to uncover 63 previously unknown duplicate Oracle Bone pairs that had remained overlooked for decades. Extensive experiments also confirm that OBD-Finder achieves competitive retrieval accuracy while offering substantially superior efficiency and interpretability compared to existing approaches. Code and deployed system are publicly available: https://github.com/cszhangLMU/OBD-Finder/.

Developing enterprise customer-service assistants with Reinforcement Learning (RL) critically depends on high-fidelity user simulators and well-aligned reward functions, since real-customer interactions are impractical for large-scale training and rule-based rewards fail to capture complex business requirements. We present CoEvoLoop, a framework that co-evolves the assistant policy together with these RL components through an expert-calibrated, multi-round loop. Beyond an LLM-based simulator for rollout generation and a multi-dimensional grader for rubric-based evaluation of service quality and compliance, CoEvoLoop integrates an expert annotation interface where sampled trajectories and grader judgments are reviewed and corrected. Each round, expert feedback updates the simulator and grader, after which the assistant is optimized with trajectory-level RL against the updated evaluation signal; monitoring interfaces support rollout inspection, dimension-wise failure attribution, and cross-round tracking. In an industrial evaluation, the system improves multi-turn service quality and reduces compliance violations over training rounds; component ablations and blind expert evaluation confirm that co-evolving the simulator and grader is load-bearing for these gains. We release the system demo code, along with a demo video.

As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation platform becomes critical. However, current evaluation pipelines remain highly fragmented and tightly coupled, hindering reproducibility and causing redundant engineering. To address this, we introduce AgentCompass, an open-source, lightweight, and extensible platform for evaluating LLM-based agents. AgentCompass organizes the evaluation process around three independent components, namely Benchmark, Harness, and Environment, thereby enabling flexible configurations without requiring the reimplementation of complex execution logic. Furthermore, it features a fault-tolerant asynchronous runtime and comprehensive trajectory analysis tools to transparently diagnose nuanced failure modes like reward-hacking. Natively supporting over 20 benchmarks across five capability dimensions, AgentCompass provides the community with a scalable and reproducible infrastructure for advancing agent research.

When an LLM agent controls real money, prompt injection stops being an information risk and becomes a financial one—and its hardest form is the confused deputy: paying is the legitimate job and only the recipient or amount is attacker-supplied, so an instruction-level safeguard has nothing to refuse. Across five models (Claude, Qwen, MiniMax) a normally-built agent is driven to a forbidden money action on up to 87% of such scenarios—and a hardened refusal prompt still loses up to 84%. Like prior work on policy enforcement outside the model (e.g. CaMeL), we take the model off the money path rather than trust it to refuse; we contribute the first open, installable, end-to-end realization of this for agent payments: BNB Agent Studio, an open-source toolchain (CLI, Python SDK, MCP server) that takes a developer from an empty directory to a deployed, self-funding LLM agent in six commands. Its two-layer, keyless-edge architecture keeps signing as fixed code isolated from the model context, prices deterministic, and spend guardrails enforced below the LLM: an identical agent fooled at a comparable rate moves funds on 0/45 everywhere, and a live testnet deployment confirms escrow, pricing, key-access, and suspension on-chain. The toolchain implements the Agent Lifecycle Protocol (ALP), a vendor-neutral draft spec composing ERC-8004 identity, ERC-8183 task commerce, x402 payments, and A2A. It is on PyPI (pip install bnbagent-studio, Apache-2.0), with a screencast at https://youtu.be/yjdvN8vE37A.

Multilingual NLP research increasingly involves hundreds or thousands of languages across different datasets. Managing, discovering, and reporting language metadata becomes a common hurdle at these scales. We present QQ, a metadata toolkit and browser explorer. QQ compiles language metadata sources into a graph of language varieties, scripts, regions, identifiers, names, and relations, and exposes it through a Python API, a command-line interface, and a browser-based explorer. Users can normalize identifiers, retrieve metadata, traverse relations, and discover which external resources contain a language. We demonstrate QQ on three workflows: an audit of the HuggingFace Hub, linking resources that use different identifier systems, and generating reproducible language-reporting tables. QQ supports FAIR-oriented metadata practices through versioning, open formats, and reusable interfaces.

We present a web demo for exploring a large-scale disambiguated knowledge base materialized from a large language model (LLM). GPTKB 2.0 contains 38.4M triples over 1.6M canonical entities, together with 207.6K consolidated relations and 66K consolidated classes. Unlike prior LLM-derived knowledge bases that largely identify entities by surface strings, GPTKB 2.0 performs context-guided disambiguation during recursive KB construction, separating homonyms and merging synonymous mentions as facts are elicited. The demo makes this process inspectable: users can browse entities, follow links across the KB, and audit the provenance of individual facts, including surface forms, candidate matches, source triples, and disambiguation decisions. The interface further supports structured SPARQL queries, natural-language questions translated to SPARQL, and entity linking from user-provided text to canonical GPTKB 2.0 entries.

Repository-scale refactoring requires coding agents to propagate a single change across many interdependent files without altering pro- gram behavior, yet to our knowledge no existing harness isolates the design choices that determine agent success on this task. We present RefactorPlatform, an open-source evaluation harness that holds the environment fixed and varies each design axis explicitly: model backbone (via OpenRouter and GitHub Copilot CLI), execution regime (baseline, retrieval-augmented, and multi-agent), and prompt specificity. Each run executes in an isolated workspace with live terminal streaming, per-task logging of tokens, diffs, and transcripts, AST-based verification, and exportable telemetry for audit and reproduction. Demonstrating the platform on 100 multi-file RefactorBench tasks across four model families, we illustrate the analyses it supports: AST-aware chunking outperforms naive token-window chunking by 25-30 percentage points (pp) across prompt modes, whereas naive retrieval falls below the retrieval-free baseline; a lean retrieval-augmented single agent (86%) beats the sub-agent configuration we evaluated (66%) on matched tasks with no task passing under delegation that fails under retrieval; and retrieval’s accuracy gains absorb its token overhead, leaving cost per successful refactoring unchanged. RefactorPlatform is open-sourced to make refactoring-agent evaluation reproducible and auditable.

Large language models are increasingly used in social-science and market research to stand in for human respondents, exploring questions ahead of fieldwork or piloting instruments. Existing systems address one facet at a time (a population’s demographics, an individual’s profile, or an answer distribution elicited from a single prompt), and few serve users without social-science training. We present Alyas, a research-simulation platform in which an agentic methodologist assistant guides the user from a research goal to a study design, laid out as a branching graph of research artifacts, and simulates its field phase over a synthetic population grounded in institutional statistical sources. On held-out Italian and US microdata, the generated populations reconstruct demographic structure the engine was not given, and, on the Italian microdata, profile-conditioned respondents answer close to the real subgroups they simulate.

Knowledge distillation (KD) is widely used to compress and post-train large language models (LLMs), yet many existing frameworks execute teacher inference with the same training-oriented backend as student optimization, leading to suboptimal efficiency. In this paper, we propose KDFlow, a novel framework for LLM distillation that features a decoupled architecture and employs SGLang for teacher inference. KDFlow combines SGLang for teacher inference with PyTorch FSDP2 for student optimization, allowing each model to run on a backend tailored to its workload. To enable efficient full-vocabulary distillation in this decoupled architecture, KDFlow transfers the teacher’s final hidden states via Ray’s object store and recomputes teacher logits on each student worker using a frozen copy of the teacher’s output head. Furthermore, our framework supports both off-policy and on-policy distillation and incorporates cross-tokenizer algorithms through highly extensible and user-friendly APIs. Experiments show that KDFlow achieves a 1.44× to 6.36× speedup over MS-SWIFT in off-policy distillation and a 1.43× to 1.75× speedup over verl in on-policy distillation. KDFlow further scales to 64 GPUs, achieving 3.68× and 2.52× strong-scaling speedups in two representative model configurations. The code and documentation are publicly available. The code and documentation are publicly available.

Large language models (LLMs) offer promising clinical decision support but remain vulnerable to hallucinated facts, unsupported recommendations, and citation errors. We present DIASENTINEL, a fully on-premise multi-agent system for one-year type 2 diabetes mellitus (T2DM) risk screening and guideline-grounded report generation from electronic health records (EHRs). The system integrates calibrated risk prediction, deterministic clinical signal extraction, Reciprocal Rank Fusion over American Diabetes Association (ADA) guidelines, and a hybrid verification layer combining rule-based checks with LLM entailment. The demonstration provides a real-time batch-screening dashboard and an interactive patient report interface with cited recommendations, verification results, and raw EHR comparison. DIASENTINEL demonstrates a practical framework for reliable, auditable, and privacy-preserving LLM-based clinical decision support. Our system demonstration can be found at https://diasentinel-demo.onrender.com/

We introduce ZServe, an adaptive framework designed to optimize the deployment of large language models (LLMs) in production environments. ZServe tackles critical challenges in LLMs serving with three key innovations: (1) Resource-aware Parameter Mining, which automatically identifies optimal hardware configurations for various accelerators, eliminating the need for manual tuning while maximizing performance; (2) Task-aware Dynamic Reconfiguration, which seamlessly switches between SLO-centric and throughput-centric serving modes based on real-time workload characteristics without service disruption; and (3) A Unified Monitoring and Scaling Interface, which simplifies deployment management across heterogeneous infrastructures. In contrast to static frameworks like RayServe and Xinference, ZServe introduces dynamic runtime adaptation, leveraging novel queue handoff and memory-efficient state migration to switch serving modes without dropping in-flight requests. Experimental results with LLMs from 7B to 70B parameters show that ZServe achieves up to 45% higher throughput during peak periods while preserving latency targets during moderate traffic, demonstrating scalability across single-node and multi-node deployments.

Retrieval Augmented Generation (RAG) is an important aspect of conversing with Large Language Models (LLMs) when factually correct information is important. LLMs may provide answers that appear correct, but could contain hallucinated information. Thus, building benchmarks that can evaluate LLMs on multi-turn RAG conversations has become an increasingly important task. Simulating real-world conversations is vital for producing high quality evaluation benchmarks. We present RAGAPHENE, a chat-based annotation platform that enables annotators to simulate real-world conversations for benchmarking and evaluating LLMs. RAGAPHENE has been successfully used by approximately 40 annotators to build thousands of real-world conversations.

Collective behavior forecasting usually relies on historical outcome traces: past sales, past votes, or prior campaign performance. Such history-side records are useful when comparable cases exist, but they are brittle for new products, emerging political actors, or settings where historical patterns fail to generalize. We propose a complementary response-side approach: instead of only extrapolating from past aggregate outcomes, we prompt large language models as third-party domain analysts to infer collective outcomes from structured context and behavioral signals. We introduce EBI, an Exogenous–Behavioral–Inference framework that represents each task through external context, behavioral signals, and the target aggregate outcome. Across three real-world domains — health supplements, elections, and crowdfunding — we preliminarily evaluate whether response-side inference is competitive with history-side baselines, which are restricted to historical outcome data. We present the EBI system, a web-based tool designed for users with limited access to historical data, survey infrastructure, or analyst resources, allowing them to explore domain-specific EBI predictions, run live inference across multiple models, and ablate exogenous and behavioral features interactively to understand prediction drivers.

We present SCOPE, a multi-agent system for profiling news media outlets with respect to bias, factuality, and credibility. Given an outlet URL, SCOPE coordinates specialized agents for article acquisition, content analysis, external evidence search, scoring, and report generation. The system follows an MBFC-style methodology while making intermediate evidence and agent outputs inspectable through a structured demonstration interface. On a benchmark of 200 media outlets, SCOPE outperforms LLM-only, article-only, search-only, and media-background-check baselines on most metrics, achieving the lowest bias and factuality MAE, perfect failed-fact-check detection, and the lowest factual error rate. These results suggest that decomposed, evidence-grounded agent workflows can support more transparent and reliable media profiling. Our code is available at https://github.com/mbaisbay/SCOPE, and a live demonstration at https://scope-demo.streamlit.app/.

Tool-using language agents are flexible but non-deterministic: an orchestrator LLM re-plans on every request, adding latency, cost, and paths that are hard to audit. We demonstrate TraceForge, an on-premise system that observes how an agent actually solves a recurring task and, on operator request, hardens it into a deterministic, immutable workflow DAG. TraceForge attaches to the OpenTelemetry traces an agent already emits (zero code changes), reconstructs each run’s execution DAG, and runs a human-gated loop: a statistical aggregation gate (proposes a DAG only once enough runs agree), a Judge-LLM reflection step, and a human freeze gate that writes a versioned, immutable specification with lineage and rollback. The pipeline is topology-agnostic—applied to single-agent tool-callers and multi-agent orchestrators alike—and all components run on-premise with no external dependencies. On held-out requests, frozen workflows reproduce the dynamic agent’s outputs exactly while removing every planner-LLM call and cutting latency 1.39x. The entire loop is driven from a browser console with no coding, and is shown end-to-end in the accompanying screencast.

Tool-integrated LLMs retrieve information, perform computations, and take real-world actions, but their reliability depends on both tool-use accuracy and intrinsic tool accuracy, including tool correctness, stability, and safety. While prior work primarily emphasizes tool use, intrinsic tool accuracy remains underexamined. We introduce OpenTools, a community-driven and maintainable toolbox for discovering, using, evaluating, and contributing open-source tools. OpenTools standardizes tool interfaces, converts documented Python functions into reviewable bundles, supports maintainer-triggered evaluation, and combines non-executing risk inspection with optional advisory LLM review. A public web demo allows users to run tools and agents, inspect evidence, contribute tests, and submit tools for maintainer review, while MCP enables controlled access from external applications. Experiments show that community-contributed, task-specific tools yield relative gains of 6% to 22% over an existing toolbox across multiple agent architectures, highlighting the importance of intrinsic tool accuracy. OpenTools is released under the Apache 2.0 license, with a demonstration video at https://www.youtube.com/watch?v=XhRwATwIBxU.

Natural Language Processing methods have enabled novel solutions and advances in the field of privacy, particularly in the sub-domain of *text-to-text privatization*, where the goal is to transform a sensitive input text into a privatized output by ideally masking (in)directly identifiable or otherwise private information. The evaluation of text-to-text privatization, however, is not straightforward, and the extant literature has utilized a myriad of techniques and metrics to quantify the privacy-preserving capabilities of privatization methods. Seeking to unify the evaluation of text-to-text privatization, we introduce PrivBench, a holistic and modular benchmarking platform for researchers and practitioners working on text privatization. PrivBench is holistic in that it evaluates privatization on a series of defined desiderata, which are structured into *modules*. PrivBench is not only modular but also extensible, allowing for future updates and benchmark versions. PrivBench is user-centered and promotes competition via real-time evaluation and a live public leaderboard. The platform is free to use and openly accessible at https://privbench.com/.

Wikipedia plays a key role in shaping public understanding of science, and its openly accessible revision history is a unique record of how scientific knowledge evolves over time. Yet scientifically meaningful revisions are obscured by the sheer volume of routine edits, leaving each article’s scientific history hidden. We present WikiSTAR (Scientific Tracking of Article Revisions), an interactive system for exploring scientifically meaningful changes across an article’s revision history. Using an LLM classifier with an expert-designed multi-label taxonomy, WikiSTAR first tags edit types such as the addition of technical terms, new research findings, and changes in scientific narrative. Then, through interactive views, an article’s full revision history can be traced at any granularity - from aggregate trends that reveal when and in which sections scientific content was added or refined, down to individual edits - showing how scientific knowledge develops at a scale previously impossible. In a user study, experts from three domains found that WikiSTAR surfaced new patterns and research questions and enabled previously impractical analyses. We release our code and a human-annotated benchmark.

While memory systems are essential for agent architectures, pervasive architectural fragmentation restricts systematic research. Existing implementations typically couple different stages of the memory lifecycle, entangle evaluation logic with specific datasets, and provide limited support for the management of heterogeneous memory types. We introduce MemTools, an interoperability research framework that decouples memory system components from their underlying deployment environments. MemTools standardizes the memory lifecycle through declarative data contracts, enabling the interchangeable assembly of components across different systems. It orthogonally separates benchmark datasets from execution protocols to facilitate controlled assessments. Furthermore, MemTools provides a unified computational interface for coordinating symbolic, neural, and multimodal memory representations within a shared runtime. Empirical evaluations on cross-system component integration, evaluation protocol reconfiguration, and heterogeneous memory coordination demonstrate that MemTools enables systematic isolation and analysis of memory design variables. These findings suggest that MemTools provides a practical and extensible infrastructure for advancing principled research on agent memory.

We present string2string Studio, an in-browser workbench for string-to-string analysis across natural language processing, computational biology, and the digital humanities. The system integrates six modules (alignment, distance, similarity, search, generation metrics, and BLAST homology search) and operates at character, word, token, line, and residue levels. Its C++ algorithms compile to WebAssembly, so core operations run locally by default without installation or data upload. The interface reports scores with their evidence (alignments, edit paths, metric matches, search hits, and homology traces), making methods inspectable, debuggable, and comparable on shared inputs. Internal benchmarks show speedups of up to 2,500x over the Python predecessor, faster global/local alignment than a general-purpose native C aligner, and exact agreement with independent references under declared settings. For homology search, the scoped client-side blastn path closely matches NCBI BLAST+ rankings and statistics under matched parameters. A curated showcase and Learn mode package canonical algorithms and metrics as reusable demonstrations. string2string Studio is open-source and freely available at string2string.org.

This paper presents Murano, an open source framework for designing, running, and reproducing mechanistic interpretability studies of large language models, intended for researchers across disciplines. These studies often combine loading, recording, attribution, intervention, and evaluation, while existing libraries tend to focus on different parts of this workflow. As a result, researchers using several libraries may need to adapt outputs from one for use by another. To bridge this gap, Murano represents operations from these five areas as composable steps. Steps exchange named result artifacts and declare the inputs they require and the outputs they produce. A pipeline executes its steps in the order supplied, and Murano uses canonical addresses when component identities pass between operations. Murano builds on existing interpretability and machine learning libraries. We demonstrate Murano through two reproductions of established interpretability studies and one illustrative sparse autoencoder case study.

We present _Anamnesis_, an interactive system for demographically controllable survey simulation using large language models. Open-source and designed for _non-technical_ users and researchers, Anamnesis enables the prototyping and stress-testing of survey instruments on virtual populations. The platform operationalizes the recently introduced Anthology and Alterity frameworks, which use structured narrative backstories to condition model responses, within a unified web interface. It supports open-ended generation, probabilistic demographic resampling, and multimodal (image and audio) surveys. We evaluate the system through two case studies: (1) replicating segments of Pew Research Center’s American Trends Panel (ATP) on political typology and biomedical issues and (2) emulating human preference in the New Yorker Caption Contest. In both cases, _Anamnesis_ produces opinion distributions that more closely match real-world survey data than standard persona-prompting baselines, offering a transparent, reproducible, and open-source alternative to proprietary simulation services.

We present **CALIBER**, an interactive dashboard for calibrated, LLM-assisted behavioral interview evaluation. The system is built around three core design principles. First, a structured **Annotate–Rubric–Rate (ARR)** workflow sequences each evaluation into span annotation, rubric consultation, and scoring, decomposing a holistic judgment into auditable steps. Second, LLM-generated STAR span highlights serve as **attention scaffolds** rather than scores: the model surfaces structural cues in the transcript while all final ratings remain the annotator’s responsibility. Third, a 21-step guided training tour with Behaviorally Anchored Rating Scales (BARS) brings every evaluator to a shared baseline before live annotation begins. On the backend, a shrinking-pool randomizer with atomic DynamoDB counters guarantees uniform item coverage across crowdworkers, and a rich telemetry stream captures click, mouse-movement, and timing data alongside every annotation. A user study (N=24) confirms that the ARR design achieves its goals: both AI-friendly and AI-averse annotators complete tasks with less effort under AI assistance, and alignment with a held-out LLM reference improves for all groups. We release CALIBER as an open-source toolkit together with a live demo and code.

Engaging in counterarguments (CAs) has been shown to be a promising approach to effectively support the development of critical thinking skills in educational settings. Developing an argumentation tutoring system that automatically provides personalized feedback on CAs would benefit both teachers and students by mitigating the constraints imposed by limited instructional resources. However, the development of such systems remains underexplored. To address this gap, we develop CAT, a CounterArgument Tutoring system that analyzes the underlying logical structure of CAs and provides tailored feedback accordingly. To evaluate CAT, we conduct a user study with 39 undergraduate students over two weeks in a real classroom, comparing it against an existing argumentation tutoring system as a baseline. The results indicate that CAT supports students in critically evaluating opposing arguments, considering alternative perspectives, and refining their own reasoning. Statistical analysis further shows that CAT elicits more engaged interaction than the baseline system. Moreover, students perceive CAT as easy to use and valuable for improving both their critical thinking and argumentation skills.

Recent progress in language models has been defined by scale, yet many practical applications benefit more from robust reasoning than extensive parametric knowledge. We introduce OCC-RAG, a family of models for faithful context-based QA, which prioritizes task-specific reasoning capabilities over knowledge capacity. To train OCC-RAG, we implement a novel pipeline synthesizing over three million multi-context, multi-hop QA examples that target multi-hop reasoning, strict context faithfulness, and calibrated abstention. Mid-trained on this corpus, the models produce structured reasoning traces with source citations grounded in literal quotes from the context. Despite their compact size, OCC-RAG models outperform their same-size general-purpose counterparts on every benchmark and exceed models 2–6× larger on financial reasoning (TAT-QA) and faithfulness (ConFiQA), attaining the best results among all evaluated open-weight models up to 32B parameters. On ConFiQA they rank behind only a frontier proprietary models; on multi-hop reasoning (HotpotQA, MuSiQue) and refusal (MuSiQue-Un) they are on par with 8B-class models and close much of the remaining gap to the largest evaluated systems. We release both our code and family of models.[<https://hf.co/collections/occ-ai/occ-rag>]

Post-implant telephone follow-up requires an agent to respond naturally, complete a structured standard operating procedure (SOP), and avoid unsupported medical advice within the same call. We present SAFER, a live speech-first system for Chinese neuromodulation follow-up that exposes three capabilities in one demonstration: real-time voice interaction with barge-in, an inspectable in-call SOP trace, and guarded handling of unscripted medical or device questions. In the dual-lane architecture, user-facing speech does not wait for asynchronous state commitment, while questions assigned to the guarded branch receive fixed non-generative deflection phrases; callback-eligible clinical or safety subclasses also create a human-follow-up record. Controlled studies report a median end-to-end voice latency of 1.07 seconds, a 0.983 transcript-level completeness-accuracy field score, and 23/30 joint routing, correctness, coverage, and grounding successes on independently expert-reviewed synthetic gold-ANSWER queries. The hosted demo lets reviewers conduct a call and inspect the evolving in-call record, available evidence text, latency, and patient-facing routing behavior. These results characterize a simulated prototype; they do not establish clinical safety or real-world robustness.

Constructing causal directed acyclic graphs (DAGs) is a core step in biomedical causal analysis, yet it remains a largely manual process. Analysts must connect study variables to prior literature, evaluate uncertain causal claims, and preserve sufficient provenance for expert review. We present EviDAG, a browser-based system for authoring causal DAGs as auditable, evidence-linked artifacts from biomedical literature. Given free-text descriptions of study concepts, EviDAG creates a reproducible literature snapshot, uses an LLM-based reasoning module to generate structured pairwise causal judgments, links literature-supported judgments to verbatim evidence excerpts, and assembles the judgments into a constraint-checked graph. Each proposed edge includes confidence estimates, provenance, and a reviewable rationale. The interface supports study specification, progress monitoring, evidence review, graph comparison, adjustment-set computation, and export. In evaluations against both compact benchmark DAGs and reference DAGs derived from published literature, EviDAG achieves high edge recall on the literature-based cohort while retaining verifiable evidence trails absent from LLM-only baselines. EviDAG thus offloads the mechanical burden of causal DAG curation while making the resulting assumptions auditable, supporting the design, analysis, and interpretation of biomedical studies.

We introduce CAFE (Compound-AI Factorial Evaluation), an open-source platform that brings design of experiments to the evaluation of compound AI systems (CAIS). Such systems expose many interchangeable choices – e.g. which retriever, model, or prompt – and practitioners rarely know which of them most affects answer quality. With CAFE, a practitioner registers each swappable component of a pipeline as a factor to build a factorial design over the chosen factors, run the resulting configurations, and score the answers on a shared rubric using a configurable LLM judge together with human raters. From these ratings it attributes answer-quality variance to the components and their interactions with mixed-effects models and reports effect sizes, significance, the best configuration, cost and latency trade-offs, and judge–human reliability. Whereas existing tools mostly either search for a good configuration or score outputs in isolation, CAFE also explains which component drives quality and whether an observed difference is significant. We validate CAFE on a retrieval-augmented question-answering (QA) pipeline over the HotpotQA benchmark dataset, where it recovers planted factor effects and stays calibrated under a permutation null. CAFE is released as a Python package and as a Web application.

Automatic Speech Recognition (ASR) for railway announcements is challenging due to noisy environments and dense, entity-rich content. We introduce RailVaani, a low-latency bilingual (Hindi-English) ASR pipeline for Indian railway public address (PA) announcements, benchmarked on RailKosh – the first bilingual annotated corpus of real railway PA audio (1,513 clips: 1,000 English, 513 Hindi; 53 stations; gold transcripts and entity labels). We evaluate ASR baselines and their LoRA fine-tuned variants with database-aware (DB-aware) post-processing (PP), using WER and task score (mean of train number accuracy, platform accuracy, and announcement-type macro-F1), as task score is the more passenger-relevant metric for this domain. Our key findings: (i) on English PA, our deployed system (Whisper Small + LoRA + PP) achieves 69.8% task score at a real-time factor (RTF) of 0.75; a larger variant (Qwen3-ASR-0.6B + LoRA + PP) matches Google STT (79.7% vs. 79.3%) at RTF=1.37; (ii) on Hindi PA, our deployed system (collabora/whisper-small-hindi + LoRA + PP) achieves 60.7% task score at RTF=0.83, surpassing Whisper Large v3 (43.5%) with a 6x smaller model; and (iii) a user study (N=63; SUS-style median=90.0) indicates strong perceived usability. Both deployed systems run faster than real-time on CPU, enabling deployment without GPU infrastructure. RailVaani, RailKosh, and the RailKosh annotation toolkit are fully open-source.

LLM agent failures are difficult to debug because the step where an error surfaces is often not the one that caused it. Existing observability tools replay execution traces but provide little support for identifying the root cause or translating diagnosis into recovery. We present AgentDebugX, an open-source debugging framework that organizes debugging as a closed loop of Detect, Attribute, Recover, and Rerun. At its core, DeepDebug performs multi-turn root-cause diagnosis through global trajectory understanding, structure-guided investigation, and cross-examination. On the Who When benchmark, DeepDebug achieves the best strict attribution accuracy among the evaluated methods on both tested open-weight backbones, reaching 28.8% exact agent-and-step accuracy on qwen3.5-9b, compared with 21.7% for the strongest single-pass baseline. On GAIA, DeepDebug repairs 13 of 73 failed tasks in a single rerun, compared with 4–6 for three decoupled self-correction baselines, improving overall accuracy from 55.8% to 63.6%. AgentDebugX exposes this workflow through a Python library, CLI, web console, and installable agentic skill. It also provides an opt-in Error Hub for sharing scrubbed failure–diagnosis–repair bundles and reusing them as debugging memory.

We present CoDA, an open-source human-LM collaborative codebook development assistant that helps domain experts develop annotation guidelines for annotating new variables in unstructured text with language models. Existing language model annotation systems primarily focus on downstream classification performance, while assuming the availability of high-quality annotation guidelines, or codebooks. In practice, developing these codebooks is time-consuming and challenging, particularly for concepts that require expert knowledge. CoDA addresses this gap by collaborating with domain experts to iteratively develop codebooks through targeted review of incorrect language model predictions and incorporating expert feedback into the evolving codebook. We demonstrate CoDA on two real-world case studies: extracting health indicators from homelessness case management notes and classifying sentiment of wildlife trafficking posts on social media. Across both domains, CoDA produces codebooks that achieve downstream classification performance comparable to expert codebooks while reducing codebook development time from several weeks to two hours and requiring 5× fewer manually annotated examples. Applying CoDA to generate annotations at scale, we identified 2,410 case notes that contained health discussions or referrals and 10,196 social media posts expressing negative sentiment that may warrant investigation for potential wildlife trafficking. By accelerating codebook development, CoDA helps practitioners use language models to extract domain-specific insights from unstructured text at scale. making in high-impact domains.

Large corpora of conversations between humans and large language models (LLMs) provide a valuable record of how AI systems are used in the wild, but extracting empirical insights from them requires substantial engineering to build custom pipelines that parse multi-platform chat logs and process platform-native artifacts. We present ShareChatInsights, an interactive system that enables natural-language querying and analysis of ShareChat, a large-scale corpus of human-LLM conversations spanning five commercial assistants. An AI agent refines each request through four stages: identify, scope, model, and analyze, turning an open-ended question into a fully specified study that the system executes and exports as a runnable script. Users can configure filters to search, inspect, or export individual conversations on a separate search interface. Users can also opt to publish completed analyses to a shared community gallery. We evaluate the system on 46 requests drawn from prior literature, validate the results against independent LLM and human judgments, and present two empirical case studies. By integrating natural-language querying with reusable analytical infrastructure, ShareChatInsights lowers the barrier to large-scale empirical research on human-LLM conversations for researchers without extensive technical skills.

Large language model (LLM) agents are increasingly used for infrastructure operations on Kubernetes, yet how to test them efficiently and accurately remains unresolved. Realistic long-horizon evaluations are time-consuming and sometimes error-prone to build, since test cases are usually written from scratch even though components such as setup, state management, and verification logic are duplicated across them. We present KARMA, an open-source framework that composes reusable, parameterized operational cases into stateful, multi-stage Kubernetes workflows. Each case textually specifies a task and a required cluster state, and contains a deterministic oracle. The cluster state is preserved across stages, so long-horizon tasks can be constructed with little additional effort. KARMA also provides controls that add further complexity, namely alternating prompt modes and adversary injection. For cross-stage testing, a regression sweep reruns all previous cases and an LLM judge adjudicates the results, detecting execution inconsistencies or conflicts introduced by agents. Alongside the framework, we release a corpus of 89 cases, 38 adversaries, and 327 workflows across eight systems.

Data attribution is a promising field in interpretability that aims to explain model behavior through the influence of its training data, with applications including debugging undesirable model behavior and training dataset curation. However, significant engineering effort is required to perform it at scale, and many cutting-edge techniques lack open-source tooling and support. Bergson is an open-source library that closes this gap, providing a host of techniques that scale to very large language models and pre-training datasets. The library natively supports on-disk gradient stores and multi-node distributed training, and provides quality-of-life tools for researchers. Finally, Bergson includes the first open-source implementations of three leading data attribution methods: MAGIC, SOURCE, and TrackStar. The library is available at https://github.com/EleutherAI/bergson.

Tree search algorithms enable systematic exploration of the proof space in neural theorem proving. Existing LLM tree search libraries primarily target natural language reasoning and do not provide native integration with formal verifiers, while theorem proving systems often rely on task-specific search implementations. We introduce TreeThink, an open-source Python library for modular, fully asynchronous tree search in neural theorem proving. It integrates established tree search methods with vLLM-based inference pipelines and diverse node evaluation techniques, ranging from lightweight heuristics to neural evaluators. We support Lean 4, Rocq, and Isabelle/HOL alongside natural language. It connects directly to each language’s Read-Eval-Print Loop (REPL) server for real-time verification and proof state extraction. We evaluate TreeThink on miniF2F and MATH500, demonstrating cross-language formal proof search, natural language reasoning support, and up to 8.0× wall-clock speedup from asynchronous execution. Source code is released under the MIT license at https://github.com/GGLAB-KU/treethink, and the library is accessible as a downloadable package at https://pypi.org/project/treethink/.

In university academic advising, identical questions can require different answers depending on a student’s department, admission cohort, and degree program, causing profile-blind retrievers to surface plausible but inapplicable evidence. We present proFILL, a method for transforming hoBIT, our college’s current rule-based advising chatbot, into a profile-aware retrieval-augmented generation (RAG) system. Rather than requiring a complete user profile upfront, proFILL progressively acquires only the profile attributes needed for each query, guided by both the query intent and the initially retrieved evidence, and uses them to condition retrieval over a profile-aware index. Extensive experiments and a human preference study show that proFILL outperforms diverse RAG baselines, is preferred by target users, and remains effective with open-weight models for cost-effective on-premise deployment.

Matching enterprise technology demands with academic resources is critical for technology transfer, yet difficult because enterprise requests are often vague, constraint-sensitive, and expressed differently from scholarly outputs. Existing search and retrieval-augmented generation (RAG) systems may retrieve topically relevant results but often provide unsupported recommendations or insufficient evidence. We present TechMatch, an interactive system for evidence-constrained matching of enterprise demands to papers, scholars, patents, and projects. Its key novelty is a trustworthy workflow that combines demand clarification, structured constraint extraction, graph-augmented retrieval, evidence-aware reranking, and traceable report generation. Each recommendation is grounded in retrieved source evidence, reducing hallucinated rationales and exposing unresolved constraints and feasibility risks. To support efficient interaction, TechMatch uses hierarchical scholar representations, lightweight profile indexing, parallel reranking, and context-budgeted evidence selection. We construct a large-scale academic corpus covering four million research resources. On a public-source benchmark, TechMatch outperforms standard RAG by 9.9% on P@5, 10.1% on NDCG@5, and 9.3% on evidence support, while enabling users to inspect evidence and generate actionable technology-transfer reports.

We present SENTINELEDGE, an open-source streaming detector for English-language scam phone calls that scores speech sentence-by- sentence and raises alerts while a conversa- tion is in progress, protecting callers rather than explaining losses afterward. It targets multi-minute scripted fraud calls, where sys- tems are closed and un-auditable and aca- demic alternatives use neural backbones un- suited to on-device budgets. SENTINELEDGE uses a streaming framework with per-sentence scoring, exponential moving average smooth- ing, and threshold-triggered alerts. It com- pares classifier heads using hand-crafted fea- tures, TF–IDF, SVM, gradient-boosted trees, and fine-tuned transformer on identical inputs and robustness conditions, revealing storage– accuracy trade-offs. We report bootstrap con- fidence intervals, stress-test ASR-style noise, adversarial scam phrasings and retraining, and zero-shot transfer to SMS and phishing URLs. Code, models, evaluation harness, and demon- stration are released under the MIT license

Linguistic style is central to how texts convey meaning and to tasks from register analysis to authorship verification, yet extracting detailed stylistic features at scale is hard: rule-based taggers are interpretable but slow, while neural style embeddings are fast but opaque. We present Neurobiber, a RoBERTa tagger that predicts 96 interpretable features from Biber’s Multidimensional Analysis in a single forward pass, bringing this analysis to corpus scale: a one-billion-token corpus tags on a single GPU in about 9.5 hours, over an order of magnitude faster than single-thread taggers. Neurobiber is trained on BiberPlus, a new open-source rule tagger that computes the same 96 features deterministically from explicit rules and ships built-in analytics such as PCA and factor analysis; users tag at scale with Neurobiber and fall back to BiberPlus for exact, auditable counts. Neurobiber reproduces BiberPlus at 0.97 macro-F1; its features recover Biber’s primary dimension of variation, and on PAN 2020 authorship verification they contribute signal complementary to a strong neural encoder. Both tools are released openly, with a live demo.

In our interdisciplinary, collaborative research, we have found that humanities and social science researchers would like to analyze data underlying LLMs as well as study how language tools (e.g., emotion detection) work on their data. Existing tools are often unintuitive for such researchers and do not provide them enough agency. In this paper, we introduce CorpusLens, an LLM-based tool to aid social, humanistic, and language science researchers in exploratory textual data analysis. We present the design of CorpusLens as well as a user study showing that researchers found CorpusLens to be more helpful than prior tools they had used, while also rating it as intuitive, easy to use, transparent and enabling.

Annotating discourse structure in multi-party dialogues is a cognitively demanding task that requires specialized expertise, yet existing annotation tools are either fully manual or provide only limited assistance. We present MUDIPA, a web-based, human-in-the-loop platform for discourse graph annotation in multi-party dialogues, built around two pluggable AI backends: a suggestion engine that produces confidence-ranked arc proposals, and an LLM-based reasoning engine that generates on-demand natural-language rationales for candidate attachments and relation labels. In a counterbalanced, within-subjects study with eight annotators, we compare AI-assisted annotation with MUDIPA against manual annotation. The AI-assisted condition achieves higher inter-annotator agreement on relation labels, higher accuracy against the gold standard, and lower perceived workload, while requiring only a modest increase in annotation time. These findings indicate that the principal value of LLMs in discourse annotation lies in supporting human decision-making, improving both annotation quality and inter-annotator agreement relative to unassisted annotation.

Tool-using LLM agents are mostly evaluated assuming all tools work. When a tool times out, returns a week-stale value, or has its description poisoned in deployment, the developer needs a controlled way to reproduce the failure, test a fix, and confirm the fix worked before deployment. We present AgentCheck, an open-source web workbench that turns an MCP server into an intervention surface. AgentCheck runs an agent against its real tools and records every tool response, then re-runs the agent with the response perturbed by a fault (12 types) injector. Matching tool calls are replayed from cache, and later tool calls go live after the agent diverges. This yields a reproduce-intervene-confirm loop: the developer toggles a mitigation, re-runs against the identical fault, and sees if the failure goes away. Scoring has two parts: deterministic pass/fail rules, plus an LLM judge for interpretive labels, validated against human annotations. Across five agents, the best passes 105/120 scenarios and the weakest only 77. The failures are usually silent, confident use of incorrect tool outputs rather than crashes. On the weakest agent, a retry mitigation raises success on timeout error faults from as few as 30% of cases to 100%, whereas stale-data faults remain near 3-4 of 10 regardless of the mitigation. AgentCheck makes these failure modes reproducible, comparable, and verifiable before deployment.

Guided Imagery and Music (GIM) organizes verbal preparation, guided listening, imagery exploration, and reflection into a structured, music-centered process. Existing LLM-based well-being agents remain text-centered, and emotion-aware music systems operate outside such multi-phase workflows, conditioning music on static mood descriptions. We present GIMFlow, an interactive, non-clinical GIM-inspired system that translates reflective dialogue into a structured musical trajectory through four phases: Prelude, Induction, Music Imaging, and Postlude. GIMFlow estimates current and target affective states in continuous valence-arousal space, constructs a deterministic four-waypoint transition, and maps waypoints to sections of an LLM-generated symbolic music blueprint for deterministic rendering; the system also supports metadata-based retrieval and unified session-level logging. A formative within-subject study (N=23) of the initial prototype revealed that generated music captured only the endpoint state; we then aligned waypoints with intra-piece sections and conducted a preliminary within-subject evaluation of the refined system (N=10). Results show descriptively favorable trends in valence and negative affect while positive affect and arousal are mixed; all findings are exploratory. GIMFlow is released as an open-source, installable research demonstration with example sessions and reproducible logging utilities.

Empirical legal research often relies on turning research questions into structured data extracted from large collections of rulings and judgments. Designing the extraction schema and then extracting the data remain a manual, expertise-heavy bottleneck. We present schematize, an open-source multi-agent system that interactively turns a researcher’s problem statement into a validated extraction schema that can later be used for autonomous extraction. Schematize couples (i) a clarification dialogue that elicits implicit expert intent, (ii) iterative schema generation, (iii) data-grounded refinement that tests the schema against documents, and (iv) chat-based post-editing. We evaluated the system with human legal professional, introducing our novel methodology, and schematize achieves top performance in most of tested configurations. While the system is designed to be domain-agnostic and applicable to any document collection, we tailor and evaluate it on legal research problems. We release schematize as a pip-installable Python package with full documentation.

We present NotAI.AI, an explainable AI-generated text detection system. Instead of returning only a binary label or confidence score, the system shows which signals influenced the prediction and lets users inspect an attribution-based sensitivity estimate obtained by subtracting selected local contributions. NotAI.AI combines sentence-level conditional probability curvature, a neural detector score, and interpretable stylometric and readability features in an XGBoost meta-classifier. It explains predictions with TreeSHAP feature contributions and can turn the resulting evidence into a concise natural-language explanation. We evaluate the system on a category-balanced subset of RAID containing human-written, clean AI-generated, and attacked AI-generated texts. The full model outperforms variants based on individual feature families, reaching 0.9685 F1 on the held-out within-subset test split. In an automatic evaluation, two model judges rate 94.5-98.6% of generated explanations as faithful to the supplied detector evidence. The web interface (https://notai-ai.vercel.app), source code (https://github.com/Oleksandr-MB/EMNLP2026DEMO_NotAI.AI), and demonstration video (https://youtu.be/l8Nk8kdBTHQ) are publicly available.

Embedding Orrery is an open-source platform for interactive 3-dimensional visualization of embedding spaces, with native support for Sparse Autoencoders (SAEs), built for data exploration and interpretability research. Given any textual, image, or vector dataset, a single pipeline - embedding, projection, clustering, automatic topic labeling and, optionally, the collection of per-document SAE feature activations - turns it into a 3D constellation that can be searched visually, semantically, lexically, or by SAE feature names; probes can be trained, configured and scored directly in the interface. SAEs are automatically downloaded and rendered as navigable constellations in which entering a prompt highlights the features it activates. Individual features can be inspected inside a dashboard and injected into the model for causal steering. The star-cartography-inspired interface sustains up to 500,000 points, including WebGL animations and volumetric nebula shaders, on 8 GB of RAM. Embedding Orrery is released under the Apache 2.0 license. Code: https://github.com/Giacomo-De-Luca/embedding-orrery. HF Spaces: https://huggingface.co/spaces/GiacomoDeLuca/orrery-demo.

Chemistry literature synthesis often requires assembling specific findings scattered across many publications, yet existing literature-search systems primarily return ranked document lists. As a result, scientists and AI agents need to locate relevant information, trace each finding to its source, and assemble cross-paper answers manually. We present AskChem, a claim-centered infrastructure that changes the retrieval unit for cross-paper chemistry search from the paper to the source-linked claim. Available source text is converted into atomic, typed claims, each carrying a source DOI and a verbatim quote or an explicit evidence locator. Over this shared claim store, AskChem exposes complementary operational structures for search and synthesis: a corpus-induced faceted taxonomy stabilized for hierarchical retrieval and browsing, an automatically extracted evidence graph linking claims through typed relations, and an exploratory living taxonomy that situates indexed papers under scientific principles. At the frozen index snapshot, AskChem contains 2.4M claims from 147K papers, 30% processed from full text and 70% from abstracts only. The system provides a web interface, as well as REST, SDK, and MCP access for AI agents. On a 30-question AskChem-Bench pilot focused on citation traceability and paper relevance rather than factual-answer accuracy, grounding a GPT-5.5 reader in AskChem yields 100% resolvable DOIs, compared with 88.3% without retrieval, and the highest citation density among five tested systems.

Building an omni-modal foundation model means evaluating it across text, image, video, and audio. Excellent evaluation toolkits exist for each modality, but their inference engines, prompt conventions, and metric implementations are mutually incompatible, so practitioners end up maintaining separate environments for every toolchain and still struggle to compare results across them. OmniEvaluator grew out of this need in our own model development: rather than reimplementing benchmarks, it connects existing inference engines and curated evaluation libraries at a higher level, exposing four inference backends, four evaluation frameworks, and over a thousand benchmarks through a single interface. Every run is recorded as an artifact capturing the full configuration for exact reproduction, and results flow into a shared dashboard for cross-model comparison. A federated mode shares GPU inference servers across concurrent evaluations, and a built-in verifier, small enough to run on CPU, keeps its score stable across engines and prompts where rule-based scoring fluctuates under configuration mismatch, matching cost-efficient commercial LLM judges without their recurring API cost. The system, demo video, and dashboard are publicly available. (https://github.com/naver-ai/omni-evaluator, https://www.youtube.com/watch?v=4Z5VZZWyXqY, https://omni-evaluator.info)

Generative-agent systems are easier to start than to inspect. A run can contain many agents, locations, messages, commands, and model calls, yet the operator often gets either a finished replay or raw logs. That makes it hard to ask why an agent moved, test a small intervention, or package a run for another researcher. GOD is a local-first control room for agent societies. From the same browser workflow, an operator can issue targeted questions or interventions and inspect the resulting replay state. The system combines a setup wizard, Agent Studio, Map Studio, a spatial replay interface, Ask and Intervene commands, and portable experiment, map, and agent packs. Its technical contribution is the command and artifact loop: live controls and replay evidence share the same operator command model, while package contracts separate scenario, map, and profile data from local runtime state. The public release includes hosted Smallville-style and PKU replays, the open-source repository, and downloadable packs. We evaluate this path on 15 completed run slots. Across the 14 intervention runs, 78 of 84 target-agent checks recorded the commanded destination, and 169 of 182 state answers matched a saved location or action string.

Systematic reviews underpin clinical guidelines, yet their data-extraction step is a major expert-labor bottleneck bound by a protocolized workflow: two reviewers extract each study independently, an adjudicator resolves disagreements, and the team keeps an auditable record of how every value was produced. Large language models can assist with extraction, but that assistance must fit established review protocols and preserve reproducibility. We present EviStreams, a live, open-source, no-code web platform that puts review teams in control of AI-assisted extraction at three key stages: program design (a structured decomposition approved before any code runs), field specification (typed field definitions calibrated from a pilot), and extracted predictions (reviewer-blinded dual review with adjudication). Working through a form builder, a domain expert defines typed fields rather than prompts, runs extraction over uploaded PDFs, inspects every value alongside the supporting passage it came from, and resolves a reviewer-blinded dual review into an auditable consensus export. An evaluation across four clinical corpora and three frontier model families, released with the system, shows that extraction quality is shaped far more by the field specification than by the choice of model. EviStreams is live at https://evistreams.com/demo and released under Apache-2.0.

Methods for repairing safety alignment in fine-tuned Large Language Models (LLMs) are evolving quickly, but the literature is fragmented across the repair pipeline: each method is typically tested on its own drifted checkpoint, with its own repair implementation, and judged against its own set of baselines. This makes methods hard to compare or select, and calls for a framework that standardizes drift-induction, repair, and evaluation while remaining flexible enough to support new paradigms and architectures. To address this gap, we introduce SAFETUNE, a source-available library that unifies drift-induction, repair, and evaluation behind a standardized pipeline. SAFETUNE hosts four repair paradigms: weight patching, safety-constrained retraining, gradient-based unlearning, and inference-time steering, with implementations audited and debugged against their original papers. A documented intervention guide suggests a starting repair for a checkpoint’s safety-capability profile, with the full pipeline runnable in four lines of code.

Agents can create rich representations, yet these representations are rarely used as a medium for expressing user intent. We present **Agent-Bound UI**, an interaction model that separates reusable interaction structures from task-specific meanings. Agents bind task-specific meanings to interactions in reusable views and publish the resulting bindings as shared state, enabling humans and agents to understand and act on shared surfaces whose meanings vary across tasks. We implement ABRA, a framework-agnostic prototype of Agent-Bound UI, in which users and agents collaborate through chat and shared interactive canvases. We demonstrate ABRA through a representative use case and evaluate four LLM-backed agents on canvas authoring, owner-side consequence resolution, and cross-agent invocation. The code and demo video are available at https://github.com/megagonlabs/abra.

As online language education increasingly shifts toward personalized and immersive learning, scenario-based learning (SBL) offers a promising way to connect language knowledge with authentic communicative contexts. However, creating or generating immersive scenario-oriented learning materials remains challenging, since systems must interpret complex real-world situations, organize them into pedagogically coherent learning paths, adapt linguistic difficulty to learners’ proficiency levels, and support interactive learning beyond static content delivery. In this work, we develop ImmersiveCFL, a multi-agent framework that generates scenario-oriented learning paths for enhancing immersive experiences during learning Chinese as Foreign Language (CFL). Given learner-customized scenarios, ImmersiveCFL schedules a pedagogically coherent learning path with knowledge retrieved from a Chinese Education Knowledge Graph, generates multimodal scenario dialogues, and creates task-based interactive simulations with multi-skill exercises. Experimental results show that ImmersiveCFL improves difficulty control, curriculum coherence, and personalization over general LLM-based baselines. An online prototype[http://smartlearn.shuishan.net.cn:8089] further demonstrates its potential for goal-driven and interactive CFL learning.

As Large Language Models increasingly power role-playing applications, simulating patients has become a valuable tool for training counselors and scaling therapeutic assessment. However, prior work remains fragmented: existing approaches rely on incompatible, non-standardized profiles, prompts, and evaluation metrics, hindering reproducibility, fair comparison, and reuse. We introduce PatientHub, a unified and modular framework that standardizes the creation, simulation, and evaluation of LLM-based patients. Our framework provides 16 patient simulators, a graph-based orchestrator for multi-turn, multi-session interactions, and a configurable LLM-as-a-judge evaluator that supports multiple rubric types. Via our command-line interface, users can generate patient profiles, run simulations, and apply rubric-driven evaluation at the turn and session level. To demonstrate PatientHub’s utility, we compare several supported simulators under a shared interaction protocol and showcase its extensibility by prototyping a new simulator variant with minimal additional code. By consolidating existing work into a single reproducible pipeline, our framework eliminates much of the infrastructure overhead that currently fragments this research and accelerates the development of new methods. Our code and data are publicly available via https://github.com/Sahandfer/PatientHub.

How an economic shock propagates through the heterogeneous decisions of diverse economic roles and emerges as aggregate fluctuations is a central question in economics. Traditional agent-based models hard-code behavioral rules and cannot read language, while recent LLM-agent simulations often omit explicit belief formation, endogenous price formation, or mechanism-grounded state transitions, and rarely let a human step into the loop. We present WonderEcon, a configurable, LLM-driven yet economic-rule-grounded multi-agent simulation system in which households, firms, commercial banks, a central bank, and a government interact across five markets, forming a self-consistent macroeconomic loop. Every agent decides through a four-step natural-language pipeline, reading real news alongside structured data, with every step observable. A player mode, with dedicated scene and character modeling, lets a human take over one household agent, yielding a counterfactual comparison and the Economic Personality Type Indicator (EPTI). On an interest-rate hiking cycle, WonderEcon demonstrates directional plausibility with respect to six well-known economic principles across four LLMs, and 33 participants find the player system satisfying. The engine and interfaces are publicly available.

Many recurring text functions are easy to describe but difficult to implement with rules, while calling a large remote model for every input introduces repeated cost, latency, and dependency on a provider. We present compile by training, which turns a natural-language specification into a reusable neural function. At compile time, teacher models generate task-specific examples that are used to train a small adapter for a compact interpreter. The resulting function runs without the teachers and can be stored, versioned, and composed like ordinary software. On FuzzyBench-Focus, compile by training reaches 83.6% accuracy, compared with 22.4% for the previous Program-as-Weights compiler, which predicts function weights in a single forward pass. A representative cold compile takes 50.9 seconds on a B300 GPU. We deploy the compiler in a public interactive service and demonstrate compiled functions in a multi-site website helper and a language-controlled 3D avatar.

Large Language Model (LLM) agents have moved beyond generating responses to executing multi-step tasks by calling tools, observing the results, and iteratively deciding the next action. Most agent systems run on desktops or servers, which support tool use and task automation. Mobile devices are also important agent environments because they are widely accessible and contain users’ data, sensors, and daily-use applications. Existing mobile agents mainly operate smartphones through graphical user interface (GUI) actions such as tapping, swiping, and typing, which often form long, interface-dependent sequences, cannot directly access device capabilities, and make execution boundaries difficult to define. We present **PalmClaw**, an open-source agent framework that runs natively on mobile phones and manages the sessions, memory, skills, tools, and agent loop directly on the device. PalmClaw exposes device capabilities as device tools with explicit arguments, structured results, and clearly defined execution boundaries. This design enables agents to use mobile capabilities directly while keeping each action explicit and controlled. Experiments show an 11.5% relative improvement in task success and a 94.9% reduction in completion time over the strongest baseline, with lower setup burden and traces illustrating how execution boundaries are applied. Code is available at https://github.com/ModalityDance/PalmClaw.

Mechanistic interpretability seeks to understand how Large Language Models (LLMs) represent and process information. Recent approaches based on dictionary learning and transcoders enable representing model computation in terms of sparse, interpretable features and their interactions, giving rise to feature attribution graphs. However, these graphs are often large and redundant, limiting their interpretability in practice. Cross-Layer Transcoders (CLTs) address this issue by sharing features across layers while preserving layer-specific decoding, yielding more compact representations, but remain difficult to train and analyze at scale. We introduce an open-source library for end-to-end training and interpretability of CLTs. Our framework integrates scalable distributed training with model sharding and compressed activation caching, a unified automated interpretability pipeline for feature analysis and explanation, attribution graph computation using Circuit-Tracer, and a flexible visualization interface. This provides a practical and unified solution for scaling CLT-based mechanistic interpretability. Our code is available at https://github.com/LLM-Interp/CLT-Forge.

First-person assistance situations are difficult to collect with controlled timing and evidence: rare hazards should not be staged, while ordinary needs often depend on subtle visual and conversational context. We present VISTA, an interactive platform that treats scenario creation as an inspectable authoring workflow rather than a one-shot video prompt. A six-stage compiler turns a seed into typed intent, scene, event, interaction, rendering, and media artifacts; human reviewers can inspect revisions and provider requests before approving execution. We evaluate the rendered outputs, not the authoring interface. In a blinded study of 60 scenarios, 11 evaluators supplied 165 judgments. They selected the human-reviewed VISTA output as the best match to the seed in 52.1% of judgments, versus 22.4% for a one-pass seed-plus-assets baseline and 25.5% for seed-only generation. Case-balanced seed-match scores were 3.65, 3.27, and 3.22, respectively. These results support the complete workflow under the tested renderers; they do not establish interface usability, authoring efficiency, or a novel planning algorithm. The public artifact is a credential-free prepared replay with a narrated screencast.

Command-line coding agents (e.g., Claude Code, Gemini CLI) can already read and write files and sustain long sessions, yet end-to-end research still fragments across chat tools, IDEs, terminals, and writing environments, and the decisions that make it auditable are rarely preserved. We present **Dr. Claw**, an open-source workspace that wraps existing coding-agent executors in a *controllable* and *auditable* human-in-the-loop workflow rather than introducing another autonomous agent. Persistent state objects, a reusable skill library, and multi-executor coordination link human decisions to AI execution, turning planning, execution, and writing into one traceable, recoverable loop. We demonstrate Dr. Claw through an interactive three-view scenario and a failure-recovery walkthrough, and evaluate it against a bare command-line agent sharing the same backend executor, so the comparison contrasts the whole orchestration layer (task graph, state objects, and skill library) with the agent it wraps. Holding the executor fixed, Dr. Claw scores higher on research completeness while persisting an auditable, recoverable process trail. Demo access: repository https://github.com/OpenLAIR/dr-claw, released under AGPL-3.0 with GPL-3.0 upstream components.

Graph retrieval-augmented generation (GraphRAG) enhances large language models with structured knowledge, yet existing systems construct knowledge graphs in a single extraction pass, producing noisy entities and brittle retrieval. RAGU, an open-source modular GraphRAG engine, addresses this by separating extraction from consolidation: entities and relations pass through two-stage typed extraction, DBSCAN-backed deduplication, LLM summarization, and Leiden community detection. A key insight motivates a compact extractor: the skills an in-pipeline LLM needs — comprehension, extraction, reasoning over context — are language skills that grow only weakly with model size, unlike factual world knowledge. Accordingly, we train Meno-Lite-0.1, a 7 B model optimized for language skills, which outperforms Qwen2.5-32B on knowledge-graph construction (+12.5% relative harmonic mean) and matches it on English GraphRAG tasks. On GraphRAG-Bench (Medical), RAGU retrieves the most complete context at every factoid level (evidence recall up to 0.84 vs. ≤0.76) and overtakes HippoRAG 2 on synthesis tasks; on multi-hop factoid QA, the apparent HippoRAG 2 advantage is shown to be largely an answer-format artifact. RAGU is installable via ‘pip install graph_ragu‘, runs on a single GPU, and is released under Apache 2.0. Code and model: https://github.com/RaguTeam/RAGU.

AI agents are increasingly adept at tackling complex, long-running tasks. With the rapid surge of autonomous capabilities, human oversight is systematically lagging behind due to limited human-centered interfacing. Aiming to address this, we introduce AgentGUI, a user-friendly, locally hosted GUI for seamlessly observing and steering AI agents amid multiple concurrent, long-running sessions. AgentGUI features 1) rich agent trajectory visualizations, 2) effective manual and automated steering, and 3) integration with and coordination between open-source and frontier agent frameworks. A controlled user study demonstrates statistically significant reduction in the time it takes to identify key elements from agent traces (38% faster, p=0.023). In a preliminary experiment, AgentGUI’s automated drift prevention feature raises the task completion rate of small local agents by as high as 34pp across a 0.8B–9B model ladder (N=50 runs per model). AgentGUI is publicly available through its project website (https://agent-gui-project.github.io/) and open-source repository (https://github.com/eth-medical-ai-lab/agent-gui), along with a demo video (https://youtube.com/watch?v=GSDyxN1gTF0).

Oncology care operates at constant pressure of absorbing rapidly evolving evidence base in biomedicine. The American Society of Clinical Oncology (ASCO) addresses this through living guidelines, but the format introduces a new burden: any recommendation can change at any point, across multiple versioned documents. We present SENTRYLINE, a living guideline-aware clinical question answering system. SENTRYLINE retrieves guideline passages through a vectorless hierarchical RAG pipeline and returns a role-specific answer with inline citations, factual and temporal verification reports, and drift detection notes that surface when a guideline has been updated. We construct ASCOBENCH, a benchmark of 405 three-turn conversations across four question categories with gold answers from expert annotators(clinicians), and use test set to evaluate SENTRYLINE against five baselines under an LLM-as-judge framework. Experiments across three generation backbones show consistent improvements over four retrieval baselines and ASCO’s guideline assistant, with particularly strong gains on Reasoning and Role-Specific questions where multi-hop synthesis and register adaptation are required.

Mineral exploration requires integrating heterogeneous geochemical, geophysical, and geological evidence, yet existing prospectivity systems often provide only opaque scores or heatmaps. We present MineTRACE, a web-based system for evidence-grounded exploration of eight commodities: Cu, Au, Ni, W, Sn, Co, Ta, and Mn. Users can explore prospectivity maps, query locations or regions, inspect supporting evidence, and interact through natural language. A transparent expert tree, informed by geological knowledge and known deposits, combines multi-source evidence into interpretable prospectivity scores. For a new location, the conversational assistant retrieves the score and supporting evidence from the analysis pipeline and presents them in natural language. The scorer achieves spatial AUC values of up to 0.917 across different test scenarios, while end-to-end evaluation assesses query accuracy and response grounding. MineTRACE makes public geoscience data easier to access, interpret, and verify, supporting more efficient and transparent mineral exploration. Our video is available via https://geo.nlp-tlp.org/video, live demo is available via https://geo.nlp-tlp.org.