DEEPSEEK-V4.1-FLASH

A long prompt usually visits every layer before a model writes its first answer. At 100,000 tokens, a 40-layer stack makes four million layer visits before decode begins.

DeepSeek-V4.1-Flash keeps the first 20 layers as a causal encoder. They read every prompt token and leave behind the final encoder state at each position.

CED defines layer-dependent global-KV projections from that final encoder state. In the shipped CSA2 map, zero-indexed decoder layer 20 is the Full source. Later decoder layers reuse its main KV; Reindex layers refresh only the Top-K selection.

What the window costs, and what bounded replay does about it

The decoder still needs one private thing: its sliding-window state. Each layer derives that state from its own hidden states, so CED's global cache cannot supply it.

Exact decoder recovery would run 20 decoder layers across a 2,560-token prompt tail. Bounded replay instead feeds only the last 128 prompt tokens through those decoder layers under SWA truncation.

The authors report negligible response-quality impact in evaluated settings and simulate the same replay during post-training. It remains an approximation, not an equivalence guarantee.

What this buys, in layer visits

Here is a derived layer-visit toy, not a reported FLOP measurement. At the config maximum of 1,048,576 positions, a dense 40-layer pass is 41.943M visits. CED plus bounded decoder replay is 20.974M: every token through 20 encoder layers, then 128 tokens through 20 decoder layers.

That toy is 50.006% of the dense count. At 4,096 tokens it is 51.563%, and at 1,024 it is 56.250%, because the bounded decoder tail stays fixed while the prompt shrinks.

Decode gets a second benefit from the full architecture, not CED alone. With all architectural improvements combined, the report says that stretching context from 4K to 1M raises single-token decode FLOPs by about a quarter.