MiniMax's Sparse Attention Is a Retrieval System Hiding in the Model

A million tokens of context at a fraction of the attention compute. The way MiniMax got there looks less like a new attention mechanism and more like something agent builders have been doing outside the model all along. ## The mechanism Full attention is quadratic. Every token attends to every token, so at a million tokens the attention bill dominates everything else. Sparse attention is the standard answer: stop reading everything. MiniMax Sparse Attention (MSA, arXiv:2606.13392) builds on grouped-query attention and splits the work into two branches. The **index branch** is lightweight. It scores the key-value cache in blocks of 128 tokens and keeps the top k = 16 blocks, which fixes the budget at 2,048 key-value tokens per query regardless of how long the context is. Selection is shared inside each GQA group and independent across groups, so one key-value head serves several query heads off one block set. The **main branch** then runs exact softmax attention over only the selected blocks. That distinction matters: the main attention is not approximate. The approximation lives entirely in the selection. The model reads a shortlist, but it reads the shortlist carefully. ## The reframe A scorer that ranks candidates, a top-k cut, and an exact read over the survivors is architecturally the same move as RAG, vector memory, and every agent memory system in production. Score, retrieve, read. The industry keeps rediscovering that you cannot afford to look at everything, so you build an index. MSA pushed that index inside the model and learned it jointly with the weights. That is also the difference that matters to you. Your RAG index is a separate, hand-built component bolted on beside the model. This one is trained with it. What transfers to your stack is the pattern, not the kernels. You cannot import this into your agent. You can take the idea that retrieval quality, not context length, is the binding constraint. ## The numbers, as the authors report them At 1M context MSA reports 28.4x lower per-token attention compute, and with their custom kernel, 14.2x prefill and 7.6x decode wall-clock speedup on H800. Quality is reported as on par with the grouped-query attention baseline, trained on a 109B-parameter mixture-of-experts model with native multimodal training and a 3T-token budget. I have not run it. "On par" always deserves a look at which benchmarks and at what scale, and these are the authors' reported figures on their own hardware path, not something you should assume in your own system. ## The part most coverage skips My day job is estimating what algorithms cost on future hardware, so the number I find most interesting is not the capability. It is the co-design. Those speedups only materialise because the algorithm was shaped for the GPU execution path. Block granularity sized for tensor cores. A selection step cheap enough to run constantly, which they got with an exp-free top-k kernel reported at 5.1x faster than torch.topk at 128K context. The algorithm and the hardware were designed against each other, and that is doing as much work here as the attention idea. Every system that scales eventually stops reading everything and starts deciding what to read. Attention just joined the club.