Keren Zhou

Author directory

2026

Tile-based programming models, such as Triton and cuTile, aim to simplify high-performance kernel development, but their practical performance, tuning behavior, and usability remain difficult to compare systematically. We present TileBench, a controlled benchmark for evaluating Triton and cuTile on NVIDIA B200 GPUs under matched operator semantics and comparable implementation structures. TileBench contains 45 operators covering diverse AI-kernel patterns and memory/computation behaviors. Each task provides a PyTorch reference, verified Triton and cuTile implementations, standardized data-types (dtype) and input-size sweeps, default and autotuned configurations, roofline-based metrics, and profiling-guided diagnosis. Our evaluation shows that performance gaps are workload-dependent: cuTile excels on a small cluster of Tensor-Core/TMA-friendly kernels, while Triton is stronger on many irregular, streaming, and bandwidth-bound operators. We further evaluate LLM-generated cuTile and Triton kernels and find that Triton is consistently more token-efficient than cuTile under the same iterative refinement protocol. TileBench is publicly available at https://github.com/Deep-Learning-Profiling-Tools/Tilebench.
LLM agents accumulate interaction histories that exceed their context windows, motivating extensive research on memory systems. Yet most existing designs are tailored to a single scenario (such as multi-session chat or a specific trajectory format), offering little evidence that they generalize across the heterogeneous trajectories encountered in deployment. We evaluate twelve baselines spanning specialized memory systems, long-context models, and an unindexed search agent across five scenarios: single-turn QA, multi-session chat, agentic-trajectory QA, memory stress tests, and long-horizon agentic tasks. Notably, DCI-Lite, an unindexed baseline that searches raw text via grep and bounded reads, is among the strongest existing baselines. This suggests that memory performance depends partly on granting the agent active control over retrieval rather than coupling a static store to a single retrieval route. We instantiate this diagnostic insight in **AutoMEM**, a strong baseline that exposes both raw-text and indexed memory views through an agent-controlled interface. Across the evaluated systems, AutoMEM achieves the best mean cross-scenario rank.