Dims & Sums

DeepSeek · model № 004

DeepSeek-V4

A million-token model family that reduces attention compute and cache storage before it reaches the expensive parts of serving

Pro / active
1.6T/49B
Max mode: open SOTA
Flash / active
284B/13B
the price-war blade
FLOPs @1M ctx
27%
vs DeepSeek-V3.2
KV cache @1M
10%
vs V3.2 · 2% vs classic

Chapters

  1. 1The Price of IntelligenceHow DeepSeek-V4 Pro and Flash target million-token contexts with different capacity and cost profiles.
  2. 2The Compression LadderHow two compression stages shrink a long context before sparse attention chooses what to read exactly.
  3. 3The 2% CacheWhy the key-value cache reaches 2% of a classic attention baseline, and how quantization and storage tiers help.
  4. 4The Stability KitHow the model keeps a very large training run stable, from attention normalization to the optimizer update.
  5. 5Trained in FP4How the team adapts the model to four-bit arithmetic without losing the behavior it trained for.
  6. 6Winning the Price WarHow a small active model and a compressed cache change the cost of a long-context request.