Prefix Caching: When It Pays and How to Design Prompts for It
Why a repeated prompt can cost almost nothing, how vLLM and SGLang find shared prefixes, what breaks a hit, and how to structure prompts for agents and chat.
prefix-caching
2 posts
A 27B Model as Your Local Agent: What It Can and Can't Do
Real coding-agent turns replayed on a 27B model on one desk-side GPU: what the prefix cache saves, how many agents fit, how well it calls tools.