Paged Attention: Virtual Memory for the KV Cache
How serving engines hand out KV cache memory in 16-token blocks instead of reserving the maximum, how the same blocks let requests share a prompt, and what that saves in memory and time to first token.