DEEPSEEK-V4.1-FLASH

Every layer keeps a sliding window over the newest 128 tokens. DeepSeek-V4 persisted that per-layer state so a cached conversation could resume without recomputation. V4.1-Flash no longer keeps it in the persistent KV cache.

The window now lives in a host-memory pool for minutes. Global KV remains in the persistent cache for at least 72 hours. One is short-lived session state; the other is reusable history.

That distinction produces the storage reduction. Removing SWA KV from persistent storage nearly halves that footprint. The remaining global KV is roughly one quarter of V4's, making the persistent footprint roughly one eighth under identical workloads.

Why exact replay is too expensive

The compressed global cache cannot reconstruct a layer's window state. Each layer produced that state from its own hidden states.

Generic exact recovery replays L × nwin tokens. The decoder-specific CED case would run 20 decoder layers across a 2,560-token prompt tail. Those are different units from the bounded replay's 128-token input tail.

Bounded replay feeds only the latest 128 prompt tokens through the relevant layers under SWA truncation. The authors report negligible response-quality impact in evaluated settings and simulate it in post-training.

Two replay paths, and a runtime decision

Encoder replay handles a global-cache hit whose encoder window is gone. It replays the end of the cached prefix alongside the new suffix, while reusing global cache entries without overwriting them.

Decoder replay handles the decoder window needed for the first generated words. It is the path that lets prefill stop at the encoder for almost the whole prompt.

The same text can produce slightly different window state depending on where replay begins. That is an explicit approximation, and post-training sees it too.