MiniMax: Lightning Attention, Agents, and What They Tried

MiniMax, a Shanghai company also known for its video and speech products, released its first open-weight language models in January 2025. In the eighteen months since, it has used three different attention designs in its flagship models, each time explaining why. That makes it the clearest case study of a question every long-context model faces: how much of the past should each new token look at, and at what cost? This piece follows the models in order.

Text-01: lightning attention and a 4-million-token context

MiniMax-01 (January 2025) introduced MiniMax-Text-01 and its vision version VL-01: a mixture-of-experts model with 456 billion parameters, 45.9 billion active per token, trained with contexts "up to 1 million tokens" and able to "extrapolate to 4 million tokens during inference".

The way it reached such lengths is lightning attention, MiniMax's implementation of linear attention. Standard (softmax) attention compares each new token with every previous token, so its cost per token grows with the context, and it must keep every previous token's keys and values in a cache. Linear attention reorganizes the computation so the past is summarized in a fixed-size state: each new token updates the state and reads from it, at a cost that does not grow with the context. The price is that the summary is lossy: the model cannot look up an exact earlier token the way full attention can.

So Text-01 mixes the two: seven lightning layers, then one softmax layer, repeated ten times. Only 10 of its 80 layers keep a cache that grows with the context, about 40 KiB per token in BF16.

M1: a hybrid reasoning model

MiniMax-M1 (June 2025) built on Text-01 and trained it to reason with large-scale reinforcement learning, including sandboxed software-engineering environments. MiniMax called it "the world's first open-weight, large-scale hybrid-attention reasoning model", with a one-million-token context. The case for linear attention looked strongest here: reasoning models write long chains of thought, and linear layers make long outputs cheap. M1 shipped in two versions with 40,000- and 80,000-token thinking budgets, under the Apache 2.0 license.

M2: back to full attention

MiniMax-M2 (October 2025) was a different model for a different job: "for coding & agentic workflows", much smaller, 230 billion parameters with 10 billion active, and with full softmax attention in all 62 layers.

MiniMax explained the reversal in a post titled Why Did MiniMax M2 End Up as a Full Attention Model?. Its reasons, in short:

• Quality at scale. The lightning hybrid matched full attention on the benchmarks of the time, but "at a larger scale ... had clear deficits in complex, multi-hop reasoning tasks", the kind agents rely on.
• Evaluation was the bottleneck. Proving that a cheaper attention is not worse needs long, expensive evaluations on the tasks that matter, and the usual benchmarks did not reveal the gap.
• The infrastructure was not ready. Linear attention lacked mature support for things production serving depends on: low-precision state storage, prefix caching, speculative decoding. A sliding-window hybrid they also tried "wasn't good enough".
• The savings start late. Linear attention only becomes cheaper than full attention past a context length that, "in theory", is a few thousand tokens, and in practice, with optimized full-attention kernels, much more.

Their conclusion was that efficient attention "still has some way to go before it can definitively beat full attention" in industrial systems. For people who serve models, this is the useful lesson: an architecture is only as cheap as the engine support behind it. A full-attention MoE with 10 billion active parameters runs fast on every engine, with prefix caching and speculative decoding (M2 ships three multi-token-prediction modules for it); an exotic attention without those features can be slower in practice.

M2.1 to M2.7: agents, and changing licenses

M2.1 (December 2025), M2.5 (February 2026), and M2.7 (April 2026) kept M2's architecture and improved its training for coding and agents: M2.5 for "coding across 10+ languages" trained in more than 200,000 environments, M2.7 for what MiniMax calls "model self-evolution".

The licenses moved more than the architecture. M2 is MIT. M2.7 is under a non-commercial license: free for personal use, research, and non-profits, but "any Commercial Use ... is prohibited without obtaining a separate, prior written authorization from MiniMax". Two releases of the same architecture, a few months apart, can have opposite answers to "can I ship this?".

M3: MiniMax Sparse Attention

MiniMax-M3 (June 2026) is natively multimodal, about 428 billion parameters with about 23 billion active, and a one-million-token context. Its attention is the third design: MiniMax Sparse Attention (MSA), from a June 2026 paper. Every layer keeps a normal cache (grouped-query, 4 key-value heads), but in 57 of its 60 layers a small index picks, for each new token, the 16 most relevant blocks of 128 earlier tokens, plus the local block, and full attention runs only over those.

The difference with Text-01 is where the saving comes from. Linear attention saves memory by not keeping the past at all; sparse attention keeps everything, so any earlier token can still be looked up exactly, and saves time by reading only a little of it. MiniMax reports 9 times faster prefill and 15 times faster decoding than M2 at a million tokens. DeepSeek and Z.ai made the same move (DeepSeek, GLM), each with its own selection method.

M3's license is a third kind: the MiniMax Community License allows commercial use with a visible "Built with MiniMax M3" credit, and requires MiniMax's written authorization for products with more than 20 million US dollars of yearly revenue.

The models at a glance

From the papers, model cards, configs, and license files, 2026-10-06. Read the license file of the exact release you use: the Hugging Face license tags do not always match the file (M2's tag says "modified-mit"; its license file is plain MIT).

What it means if you serve them

• Text-01 and M1 need hybrid-model support: an engine that manages both a fixed-size state and a growing cache, as for the hybrid models in our hybrid attention piece. In exchange, a million-token context costs little cache memory.
• M2 to M2.7 are the easy ones: a standard grouped-query MoE with 10 billion active parameters and multi-token-prediction heads for speculative decoding. That simplicity was the point.
• M3 needs MSA kernels: check that your engine supports it, and its MXFP8 or MXFP4 checkpoints if you use them.
• Check the license per release: MiniMax's have ranged from MIT to non-commercial within one model family.

Next

Moonshot AI took DeepSeek's architecture as it was, and scaled it to a trillion parameters: the next piece follows the Kimi models.