The Open-Weight Labs

The labs behind today's open-weight models: their models, their breakthroughs, and what each changed for the ecosystem.

OPEN-MODEL LABS · 01STANDARD ATTENTION · 4.8 MiBLATENT (MLA) · 68.6 KiB71×LESS KV PER TOKEN

DeepSeek: From V1 to V4, the Models and the Breakthroughs

DeepSeek's models in order, the idea behind each one (MoE, latent attention, FP8 training, RL reasoning, sparse attention), and what each changed for everyone else.

OPEN-MODEL LABS · 02GLM-4.5368 KiBGLM-587.75 KiBGLM-5.3-FLASH11 KiB33×KV CACHE PER TOKEN

Z.ai's GLM Models: From ChatGLM to GLM-5

Z.ai's (Zhipu's) GLM models in order, from the GLM pretraining objective to agentic MoE models, the idea behind each step, and what each changed for the ecosystem.

OPEN-MODEL LABS · 03TEXT-01M2M3LINEAR → FULL → SPARSE ATTENTION

MiniMax: Lightning Attention, Agents, and What They Tried

MiniMax's models in order: lightning (linear) attention at a 4-million-token context, reasoning, the return to full attention for agents, and what each choice taught the ecosystem.

OPEN-MODEL LABS · 042.8TPARAMETERS69 KDA + 24 MLA LAYERS

Kimi: Moonshot AI's Models, from Long Context to Trillion-Parameter Agents

Moonshot AI's Kimi models in order, from long-context chat to trillion-parameter agentic MoE models, the idea behind each step, and what each changed for the ecosystem.