Articles

What DeepGEMM computes for DeepSeek's models: block-scaled FP8 and FP4 matrix multiplications, grouped GEMMs for MoE experts, and the fused Mega MoE kernel.

Why serving a big MoE model means spreading its experts over many GPUs, what dispatch and combine cost, and how DeepEP and its V2 redesign make it fast.

DeepSeek's open-source engine runs one model on Blackwell and Ascend: why it came out, what the code shows, what was kept back, and what it means for vLLM users.

Qwen's preview of its next architecture, piece by piece: linear and sparse attention, a gated four-branch residual, 512 experts and an n-gram memory, and what each costs to serve.

DeepSeek's Engram adds a hashed n-gram memory that a model looks up instead of computing: how it works, why it helps, and how DeepSeek-V4.1 and Qwen serve it.

Common load-test mistakes made on purpose next to the correct run, on one vLLM server, so each rule comes with how far it moves the numbers.

Why Kimi K2 Thinking ships in INT4 and gpt-oss and K3 in MXFP4: training with rounding in the loop, explained and measured on a small model.

Write SAXPY in CUDA, compile it for a Blackwell GPU, check it, time it, and read what Nsight Systems and Nsight Compute say about it.

Real coding-agent turns replayed on a 27B model on one desk-side GPU: what the prefix cache saves, how many agents fit, how well it calls tools.

A rotation flattens activation outliers and, on real layers, rescues INT4 activations. On Qwen3-8B in NVFP4, it cost 5 GSM8K points instead.

The parts of a GPU that decide inference speed, from streaming multiprocessors to DRAM, with one real GPU's numbers and three measurements.

GSM8K said every 4-bit format matched BF16. A noise baseline, paired comparisons, token-level KL, and a code benchmark showed which ones did not.