Measuring the Damage: Evaluations That Catch Quantization Errors
GSM8K said every 4-bit format matched BF16. A noise baseline, paired comparisons, token-level KL, and a code benchmark showed which ones did not.
evals
1 post
1 post
GSM8K said every 4-bit format matched BF16. A noise baseline, paired comparisons, token-level KL, and a code benchmark showed which ones did not.