In April 2026, DeepSeek released two related Mixture-of-Experts models. A Mixture-of-Experts, or MoE, keeps a large collection of specialist sub-networks and uses a router to activate only a small selection for each token. DeepSeek-V4-Pro has 1.6T parameters, with 49B active for a token. DeepSeek-V4-Flash has 284B parameters, with 13B active. Here, T means trillion and B means billion. Total parameters describe everything stored in the checkpoint; active parameters describe the fraction that does arithmetic for one next-token step. The shipped configs select six routed experts per token, so the model can carry a much larger library of learned patterns than it executes at once. That is the first tradeoff in V4: retain capacity in the checkpoint, then keep the routine inference path closer to 49B or 13B active parameters. Both models support a million-token context, use the MIT License, and were trained on 32T+ tokens. Those two token counts describe different things. The training total is the text the model encountered while learning, 32T tokens for Flash and 33T for Pro. The context length is the live working memory available to one request. A million-token context is not a promise that every answer should be that long. The official API documentation sets a 384K maximum output, while the model can use the remaining context for the prompt and its earlier work. The subject of this autopsy is the bridge between those scales: a very large model family trying to make a very long working memory practical to serve.
Before we use those ratios, we need the two cost units behind them. FLOPs, short for floating-point operations, count the arithmetic used to produce a token. The count rises when a model must compare the current token with more of its history. The key-value cache, usually shortened to KV cache, is the saved attention state for that history. Each past token leaves a key that later tokens can search for and a value that they can read after finding it, in every attention layer. At one million tokens, that means the system must retain and revisit a million pieces of history rather than a short chat transcript. There are two phases to keep separate. Prefill is the initial pass that reads a prompt and creates its cache. Decode is the later loop that generates one token at a time while consulting that cache. The paper's single-token inference FLOPs headline concerns the decode side; cache size constrains both the memory required after prefill and how many long requests fit on a server. In an ordinary architecture, both the math and the memory grow with every new token. That is why almost nobody ships this context length without changing how the cache is stored and searched.
How to read this autopsy
We will follow the cost of one long conversation through the model. First comes compressed sparse attention, the mechanism that decides which parts of a large history deserve detail (chapter 2). There we will separate the 4-token CSA summaries from the 128-token HCA summaries and see why one layer can be selective while another scans a coarser outline. Then comes the cache layout, including a persisted tier for reusable prefixes (chapter 3). That chapter turns the abstract cache ratio into a storage question: what stays on accelerator memory, what can be compressed, and what can be read back from disk. Chapter 4 covers the training safeguards that keep a 1.6T-parameter run stable. Chapter 5 explains four-bit precision inside the experts and why quantization-aware training matters before deployment. Chapter 6 returns to the invoice, where cache hits, cold input, output, and concurrency become distinct lines. These are separate systems, but they work as one chain: store less history, read less history, move fewer bytes, and leave more hardware capacity for concurrent requests.