Microscaling: MXFP8, MXFP6, MXFP4 and Block Scales
The OCP microscaling formats: 32 values sharing one power-of-two scale, what MXFP8, MXFP6, and MXFP4 cost in accuracy on a real model, and why the scale rule matters.
Articles
NVFP4: Two Levels of Scale
How NVFP4 differs from MXFP4: 16-value blocks, FP8 block scales, and a per-tensor scale, with what each part buys measured on Qwen3-8B.
Which GPU Computes Which Format: Ampere to Rubin, MI300 to MI355
Which number formats each data-center GPU computes natively, at what rate, from the vendors' own pages: A100 to Rubin, MI300X to MI355X.
Integers for AI: INT8, INT4, Scales, and Zero Points
How integers represent real weights: scales, zero points, per-tensor, per-channel, and per-group, measured on every weight of Qwen3-8B.
FP8: E4M3, E5M2, and Why There Are Two
The two FP8 formats, what each is for, and how FP8 is scaled, measured on Qwen3-8B's weights and activations against INT8.
FP16 vs. BF16: Same Size, Different Trade
Why training moved from FP16 to BF16, and why a BF16 model may still run fine in FP16: the same 16 bits split differently, measured on Qwen3-8B.
TF32 and Accumulation: What Tensor Cores Really Compute In
Inputs, products, and accumulators: the precision a matrix multiplication really uses, and why FP32 accumulation still matters.
Distillation: How Small Models Learn from Big Ones
What distillation is, the ways labs distill models, and how they use it on their own models and on others', from primary sources.
How a Computer Stores a Real Number: Sign, Exponent, Mantissa
What the bits of a floating-point number mean, why range and precision trade against each other, with an interactive bit explorer.
Chunked Prefill: Trading First-Token Time for Smooth Streaming
How engines split long prompts across steps, what the per-step token budget trades between time to first token and time per token, and how to set it.
Structured Output: How Engines Force Valid JSON, Token by Token
How vLLM and SGLang guarantee valid JSON or a grammar: compiling a schema into a token mask, applying it at every step, and what it costs in latency and quality.
Speculative Decoding from First Principles
How a cheap guess and one verification pass produce several tokens per step, when it speeds things up and when it does not, measured.