Engram: Memory You Look Up Instead of Compute
DeepSeek's Engram adds a hashed n-gram memory that a model looks up instead of computing: how it works, why it helps, and how DeepSeek-V4.1 and Qwen serve it.
llms-explained
3 posts
Qwen3.8-Flash-Next, Layer by Layer
Qwen's preview of its next architecture, piece by piece: linear and sparse attention, a gated four-branch residual, 512 experts and an n-gram memory, and what each costs to serve.
Distillation: How Small Models Learn from Big Ones
What distillation is, the ways labs distill models, and how they use it on their own models and on others', from primary sources.