TTFT, ITL, Goodput: Measuring What Users Feel
What TTFT, TPOT, ITL, end-to-end latency, and goodput each measure, how the tools compute them, and why peak throughput is the wrong target.
latency
2 posts
Benchmark an Inference Server Without Lying to Yourself
Common load-test mistakes made on purpose next to the correct run, on one vLLM server, so each rule comes with how far it moves the numbers.