Choosing an Inference Server: The Landscape, from Data Centers to Laptops
Inference servers in three groups: production engines and their stacks, local apps built on llama.cpp, and enthusiasts' specialized engines, with how to pick.
Inference Engines
Choosing, running, and tuning LLM inference engines in production — vLLM, SGLang, TensorRT-LLM, llama.cpp and friends.
DeepInfer: What DeepSeek Open-Sourced, and What It Kept
DeepSeek's open-source engine runs one model on Blackwell and Ascend: why it came out, what the code shows, what was kept back, and what it means for vLLM users.