Measuring the Damage: Evaluations That Catch Quantization Errors
GSM8K said every 4-bit format matched BF16. A noise baseline, paired comparisons, token-level KL, and a code benchmark showed which ones did not.
benchmarking
3 posts
TTFT, ITL, Goodput: Measuring What Users Feel
What TTFT, TPOT, ITL, end-to-end latency, and goodput each measure, how the tools compute them, and why peak throughput is the wrong target.
Benchmark an Inference Server Without Lying to Yourself
Common load-test mistakes made on purpose next to the correct run, on one vLLM server, so each rule comes with how far it moves the numbers.