Aryan Sood
Author directory2026
Dropping the Anchor: Statistical Context Summarization for Distributed Systems via Pulsar Attention
Aryan Sood | Shantanu Acharya | Gaurav Kumar Nayak
Findings of the Association for Computational Linguistics: EMNLP 2026
Aryan Sood | Shantanu Acharya | Gaurav Kumar Nayak
Findings of the Association for Computational Linguistics: EMNLP 2026
Inference with large language models (LLMs) on long sequences is computationally expensive due to the quadratic complexity of self-attention. Distributed blockwise methods such as Star Attention reduce this cost by sharding context across hosts, but rely on prepending a static, content-blind copy of the first block to every host. We propose Pulsar Attention, which replaces the static anchor with two lightweight, content-aware components: a small attention-sink prefix that stabilizes softmax, and compact cross-block summaries built via a Max-IDF heuristic that selects chunks containing globally rare tokens. This reduces the Phase 1 per-GPU FLOPs by up to 3.3× over Star Attention while retaining an identical KV cache footprint. On RULER with Llama-3.1-8B-Instruct, Pulsar Attention outperforms Star Attention at sequence lengths up to 128K tokens and remains competitive with dense attention across most tasks, with task-dependent absolute gains of up to 4.7% over the dense baseline.