Dims & Sums

DEEPSEEK-V4.1-FLASH

Three Modes, One Cache

chapter 3 of 7 · about 8 minutes

CSA2 compresses attention on three axes: the bytes in each entry, the number of tokens per entry, and the number of layers that write a cache. Four Full CSA2 layers write global main KV. Thirty-four later CSA2 layers reuse compatible KV; layers 0 and 1 are SWA-only.

Sharing a cache and sharing an attention selection are different choices. CSA2 can share the stored cache while allowing selected layers to pick a fresh subset of it. That separation is the point of its three modes.

Full modemain KVindexer Kindexer Qtop-K indiceswindow KVReindex modemain KVindexer Kindexer Qtop-K indiceswindow KVReuse modemain KVindexer Kindexer Qtop-K indiceswindow KVcomputed in this layerreused from the last Full layerreused indices, no rescoringnot computed at allevery modecomputes its ownqueries and window KV
The three CSA2 modes. Green cells are computed in the current layer, yellow cells are shared from the last Full layer, red is a reused selection. Every mode computes its own queries and its own window state.

What each mode does, and why all three exist

A Full layer computes the compressed keys and values, builds indexer keys, then makes a top-512 selection. Layers 2, 8, 14 and 20 do this.

A Reindex layer writes no cache. It scores the shared cache with its own query and selects a new 512 entries. Layers 24, 28, 32 and 36 use this mode.

A Reuse layer computes its own main query and SWA KV, then takes main KV and the latest compatible Top-K set from a preceding Full or Reindex layer. It performs no new indexing. Fifteen decoder layers work this way.

A decoder made entirely of Reuse layers would read the same 512 entries twenty times. A decoder made entirely of Full layers would store 20 fresh copies of history. The decoder uses one Full layer and four Reindex layers between those extremes.

Why the index also needed fixing

Sparse attention needs a cheap first guess about where to look. The indexer provides it. At a million tokens, scanning every visible position in every layer would erase much of the cache saving.

Decoder layer 20 scans the context once and ranks blocks of eight positions by their best score. It keeps up to 2,048 blocks, yielding up to 16,384 candidate positions.

Later Reindex layers score only that pool, then choose their own 512 entries. Their indexer cost is bounded even when the context grows.

one million positions of contextlayer 20, the decoder's first Full layer: scores all of themcandidate pool: up to 2,048 blocks × 8 = up to 16,384 positionsblocks are ranked by the best score inside them, not by a single positionlayers 24, 28, 32, 36: score only the pool, pick their own 512green: this layer's final 512 entriespale green: inside the pool but not selected this timegrey: outside the pool, never rescored, so the cost stops growing with context length
One full scan, then a bounded search. Decoder layer 20 scores every visible position and keeps up to 2,048 blocks as a candidate pool. The decoder's Reindex layers work inside that pool and each pick their own 512 entries, so their indexer cost stops growing with context length.

Make three decoder layers read the same incident

Build a pool with Full. Change the question. Then compare Reindex with Reuse.

layer's question:

toy context, 18 word groups stand in for a long document

deploygatewaycachekeychangedp95roserollbackrestoredprefixownerMinaalertresolvedtestpassedreleasenotes

context searched this layer

0

toy positions

entries given to attention

0

512 in the released model

shared cache copied?

no

all modes read one cache

no decoder layer chosen

Build a shared cache first, then try a later layer.

The toy makes the compromise visible. Full is the expensive layer that makes a shared cache and a broad pool. Reindex pays only to choose new entries inside that pool. Reuse does neither, so its latest compatible Top-K set remains until a later Reindex layer refreshes it.
The first Full layer still scans the visible context. Later Reindex layers are bounded by the candidate pool. The report says potential sparse-selection errors remain a boundary case outside the settings it evaluated.
receipts, every claim in this chapter, checked 2026-08-31