Distillation: How Small Models Learn from Big Ones
What distillation is, the ways labs distill models, and how they use it on their own models and on others', from primary sources.
training
2 posts
Quantization-Aware Training: When the Model Learns Its Own Precision
Why Kimi K2 Thinking ships in INT4 and gpt-oss and K3 in MXFP4: training with rounding in the loop, explained and measured on a small model.