NVFP4 on Blackwell: Serving a 4-Bit Model End to End
What NVFP4 stores, how vLLM serves it on Blackwell, and what it changes in memory, speed, and quality.
nvfp4
3 posts
Rotations for 4-Bit Activations: QuaRot, SpinQuant, and Friends
A rotation flattens activation outliers and, on real layers, rescues INT4 activations. On Qwen3-8B in NVFP4, it cost 5 GSM8K points instead.
NVFP4: Two Levels of Scale
How NVFP4 differs from MXFP4: 16-value blocks, FP8 block scales, and a per-tensor scale, with what each part buys measured on Qwen3-8B.