Rotations for 4-Bit Activations: QuaRot, SpinQuant, and Friends
A rotation flattens activation outliers and, on real layers, rescues INT4 activations. On Qwen3-8B in NVFP4, it cost 5 GSM8K points instead.
rotations
1 post
1 post
A rotation flattens activation outliers and, on real layers, rescues INT4 activations. On Qwen3-8B in NVFP4, it cost 5 GSM8K points instead.