How serving engines hand out KV cache memory in 16-token blocks instead of reserving the maximum, how the same blocks let requests share a prompt, and what that saves in memory and time to first token.
Why serving engines re-form the batch at every decode step, what it buys in throughput and latency, and the settings that control it.
Why a 27B model holds 262k tokens of context on a 24 GB GPU: what hybrid, sliding-window, and sparse attention store and read per token, on real models.
Compute what each token of KV cache costs from a model's config, then predict the capacity vLLM reports at startup, and see where the two differ.