https://theservingstack.com Inference Engineering, In Depth Mon, 26 Oct 2026 15:12:45 GMT https://validator.w3.org/feed/docs/rss2.html https://github.com/jpmonette/feed en https://theservingstack.com/og-default.png https://theservingstack.com © 2026 The Serving Stack https://theservingstack.com/deepgemm https://theservingstack.com/deepgemm Sun, 25 Oct 2026 07:30:00 GMT The Serving Stack gpu-for-inference deepgemm gpu quantization moe https://theservingstack.com/expert-parallelism-deepep https://theservingstack.com/expert-parallelism-deepep Sat, 24 Oct 2026 07:30:00 GMT The Serving Stack serving-in-production parallelism deepep moe sglang https://theservingstack.com/deepinfer https://theservingstack.com/deepinfer Fri, 23 Oct 2026 07:30:00 GMT The Serving Stack inference-engines deepseek deepinfer sglang ascend https://theservingstack.com/qwen38-flash-next https://theservingstack.com/qwen38-flash-next Thu, 22 Oct 2026 07:30:00 GMT The Serving Stack llms-explained qwen moe hybrid-attention vllm https://theservingstack.com/engram-conditional-memory https://theservingstack.com/engram-conditional-memory Wed, 21 Oct 2026 07:30:00 GMT The Serving Stack llms-explained engram moe deepseek qwen https://theservingstack.com/benchmarking-inference-servers https://theservingstack.com/benchmarking-inference-servers Sun, 18 Oct 2026 04:50:30 GMT The Serving Stack serving-in-production benchmarking vllm latency https://theservingstack.com/quantization-aware-training https://theservingstack.com/quantization-aware-training Sun, 18 Oct 2026 04:48:09 GMT The Serving Stack quantization qat mxfp4 int4 training https://theservingstack.com/first-cuda-kernel https://theservingstack.com/first-cuda-kernel Sat, 17 Oct 2026 22:02:32 GMT The Serving Stack cuda-for-inference cuda gpu profiling https://theservingstack.com/local-27b-agent https://theservingstack.com/local-27b-agent Sat, 17 Oct 2026 22:01:08 GMT The Serving Stack building-with-ai agents tool-use sglang prefix-caching https://theservingstack.com/rotation-quantization https://theservingstack.com/rotation-quantization Sat, 17 Oct 2026 21:56:30 GMT The Serving Stack quantization rotations nvfp4 llm-compressor vllm https://theservingstack.com/gpu-in-one-picture https://theservingstack.com/gpu-in-one-picture Sat, 17 Oct 2026 21:50:17 GMT The Serving Stack gpu-for-inference gpu cuda inference https://theservingstack.com/quantization-evals https://theservingstack.com/quantization-evals Sat, 17 Oct 2026 21:47:32 GMT The Serving Stack quantization evals vllm benchmarking https://theservingstack.com/smoothquant-w8a8 https://theservingstack.com/smoothquant-w8a8 Sat, 17 Oct 2026 21:46:02 GMT The Serving Stack quantization smoothquant fp8 llm-compressor vllm https://theservingstack.com/kv-cache-quantization https://theservingstack.com/kv-cache-quantization Sat, 17 Oct 2026 21:43:13 GMT The Serving Stack quantization kv-cache vllm fp8 long-context https://theservingstack.com/ttft-itl-goodput https://theservingstack.com/ttft-itl-goodput Sat, 17 Oct 2026 09:42:58 GMT The Serving Stack serving-in-production benchmarking observability vllm latency https://theservingstack.com/prefill-vs-decode https://theservingstack.com/prefill-vs-decode Fri, 16 Oct 2026 23:53:04 GMT The Serving Stack request-to-tokens prefill-decode gpu vllm inference https://theservingstack.com/life-of-a-request https://theservingstack.com/life-of-a-request Fri, 16 Oct 2026 23:43:01 GMT The Serving Stack request-to-tokens inference serving vllm prefill-decode https://theservingstack.com/awq-explained https://theservingstack.com/awq-explained Sun, 11 Oct 2026 09:15:49 GMT The Serving Stack quantization awq llm-compressor vllm https://theservingstack.com/gptq-explained https://theservingstack.com/gptq-explained Sat, 10 Oct 2026 13:01:28 GMT The Serving Stack quantization gptq llm-compressor vllm https://theservingstack.com/gguf-quant-types https://theservingstack.com/gguf-quant-types Sat, 10 Oct 2026 12:42:14 GMT The Serving Stack quantization gguf llama-cpp https://theservingstack.com/ptq-and-outliers https://theservingstack.com/ptq-and-outliers Sat, 10 Oct 2026 12:42:14 GMT The Serving Stack quantization ptq outliers https://theservingstack.com/bits-size-quality https://theservingstack.com/bits-size-quality Sat, 10 Oct 2026 12:42:13 GMT The Serving Stack quantization benchmark https://theservingstack.com/low-precision-pitfalls https://theservingstack.com/low-precision-pitfalls Fri, 09 Oct 2026 19:19:15 GMT The Serving Stack formats numerics debugging https://theservingstack.com/quantization-map https://theservingstack.com/quantization-map Fri, 09 Oct 2026 19:19:15 GMT The Serving Stack quantization overview https://theservingstack.com/gpu-format-support https://theservingstack.com/gpu-format-support Fri, 09 Oct 2026 19:19:14 GMT The Serving Stack formats gpu hardware https://theservingstack.com/nvfp4-explained https://theservingstack.com/nvfp4-explained Fri, 09 Oct 2026 19:19:14 GMT The Serving Stack formats nvfp4 fp4 https://theservingstack.com/microscaling-mx-formats https://theservingstack.com/microscaling-mx-formats Fri, 09 Oct 2026 19:19:14 GMT The Serving Stack formats mxfp4 microscaling https://theservingstack.com/integers-for-ai https://theservingstack.com/integers-for-ai Fri, 09 Oct 2026 19:19:13 GMT The Serving Stack formats int8 int4 https://theservingstack.com/fp8-formats https://theservingstack.com/fp8-formats Fri, 09 Oct 2026 19:19:13 GMT The Serving Stack formats fp8 https://theservingstack.com/tf32-and-accumulation https://theservingstack.com/tf32-and-accumulation Fri, 09 Oct 2026 19:19:12 GMT The Serving Stack formats tf32 tensor-cores https://theservingstack.com/fp16-vs-bf16 https://theservingstack.com/fp16-vs-bf16 Fri, 09 Oct 2026 19:19:12 GMT The Serving Stack formats fp16 bf16 https://theservingstack.com/floats-explained https://theservingstack.com/floats-explained Fri, 09 Oct 2026 19:19:11 GMT The Serving Stack formats fp32 floating-point https://theservingstack.com/distillation https://theservingstack.com/distillation Fri, 09 Oct 2026 19:19:11 GMT The Serving Stack llms-explained distillation training https://theservingstack.com/glm-models https://theservingstack.com/glm-models Tue, 06 Oct 2026 08:26:25 GMT The Serving Stack open-model-labs glm zai zhipu moe agents https://theservingstack.com/nvfp4-on-blackwell https://theservingstack.com/nvfp4-on-blackwell Tue, 06 Oct 2026 08:26:25 GMT The Serving Stack quantization nvfp4 fp8 blackwell vllm https://theservingstack.com/speculative-decoding https://theservingstack.com/speculative-decoding Tue, 06 Oct 2026 08:26:25 GMT The Serving Stack request-to-tokens speculative-decoding vllm sglang https://theservingstack.com/prefix-caching https://theservingstack.com/prefix-caching Tue, 06 Oct 2026 08:26:25 GMT The Serving Stack request-to-tokens prefix-caching kv-cache vllm sglang agents https://theservingstack.com/chunked-prefill https://theservingstack.com/chunked-prefill Tue, 06 Oct 2026 08:26:25 GMT The Serving Stack request-to-tokens chunked-prefill scheduling vllm sglang https://theservingstack.com/structured-output https://theservingstack.com/structured-output Tue, 06 Oct 2026 08:26:25 GMT The Serving Stack request-to-tokens structured-output json tool-calling vllm sglang xgrammar https://theservingstack.com/roofline-for-llms https://theservingstack.com/roofline-for-llms Tue, 06 Oct 2026 08:26:25 GMT The Serving Stack gpu roofline decode memory-bandwidth https://theservingstack.com/minimax-models https://theservingstack.com/minimax-models Tue, 06 Oct 2026 08:26:25 GMT The Serving Stack open-model-labs minimax linear-attention moe agents https://theservingstack.com/deepseek-models https://theservingstack.com/deepseek-models Tue, 06 Oct 2026 08:26:25 GMT The Serving Stack open-model-labs deepseek moe mla reasoning https://theservingstack.com/nvidia-speech-models https://theservingstack.com/nvidia-speech-models Tue, 06 Oct 2026 08:26:24 GMT The Serving Stack speech-ai nvidia nemotron parakeet canary magpie voice-agents https://theservingstack.com/disaggregated-serving https://theservingstack.com/disaggregated-serving Tue, 06 Oct 2026 08:26:24 GMT The Serving Stack serving disaggregation dynamo llm-d vllm sglang https://theservingstack.com/kimi-models https://theservingstack.com/kimi-models Tue, 06 Oct 2026 08:26:24 GMT The Serving Stack open-model-labs kimi moonshot moe agents https://theservingstack.com/asr-whisper-vllm https://theservingstack.com/asr-whisper-vllm Tue, 06 Oct 2026 08:26:24 GMT The Serving Stack speech-ai asr whisper vllm diarization realtime https://theservingstack.com/tts-landscape https://theservingstack.com/tts-landscape Tue, 06 Oct 2026 08:26:24 GMT The Serving Stack speech-ai tts voice audio-codecs https://theservingstack.com/choosing-an-inference-engine https://theservingstack.com/choosing-an-inference-engine Sun, 04 Oct 2026 21:16:16 GMT The Serving Stack inference-engines serving vllm sglang tensorrt-llm llama-cpp ollama https://theservingstack.com/paged-attention https://theservingstack.com/paged-attention Sun, 04 Oct 2026 21:16:16 GMT The Serving Stack request-to-tokens kv-cache paged-attention vllm serving https://theservingstack.com/continuous-batching https://theservingstack.com/continuous-batching Sun, 04 Oct 2026 17:24:43 GMT The Serving Stack request-to-tokens batching scheduling vllm serving