Collections
Inference Engines
Choosing, running, and tuning LLM inference engines in production — vLLM, SGLang, TensorRT-LLM, llama.cpp and friends.
2 posts
How an LLM Request Becomes Tokens
The core mental model of LLM inference, one mechanism at a time, shown with figures and widgets and checked on real hardware.
10 posts
LLMs, Explained
How large language models are built, trained, and improved, explained for engineers who run them.
3 posts
The GPU for Inference Engineers
What the hardware does during inference, and why it explains every serving technique.
3 posts
Quantization in Practice
Smaller numbers, served for real: formats, kernels, and the quality you pay.
12 posts
Serving in Production
From one engine to a service: metrics, routing, disaggregation, and the stacks that run them.
4 posts
Speech AI
Speech recognition and synthesis, served: how the models work, what serves them, and who leads.
3 posts
The Open-Weight Labs
The labs behind today's open-weight models: their models, their breakthroughs, and what each changed for the ecosystem.
4 posts
Number Formats for AI
FP32 to NVFP4: how AI hardware stores numbers, what each format keeps and loses, and which GPU computes it.
9 posts
Building with AI
Real projects built with models we serve ourselves: what works, what doesn't, what it costs, measured.
1 post
CUDA for Inference
Hands-on CUDA for inference engineers: write, compile, profile, and speed up the kernels that serving runs.
1 post