NVFP4 on Blackwell: Serving a 4-Bit Model End to End
What NVFP4 stores, how vLLM serves it on Blackwell, and what it changes in memory, speed, and quality.
fp8
4 posts
SmoothQuant and W8A8: Quantizing Activations Too
Why activation outliers break INT8, how SmoothQuant moves them into the weights, and what it changes for FP8 W8A8, measured on Qwen3-8B with vLLM.
Quantizing the KV Cache: FP8 and Below
Seven KV cache formats in vLLM, from FP8 to 3-bit, on one model: how many more tokens fit, what long-context recall loses, and why speed follows the kernel.
FP8: E4M3, E5M2, and Why There Are Two
The two FP8 formats, what each is for, and how FP8 is scaled, measured on Qwen3-8B's weights and activations against INT8.