KIMI K3

In July 2026, Moonshot published the Kimi K3 weights and called the release “Kimi K3: Open Frontier Intelligence.” Under that title is a concrete machine: a 2.78T-parameter mixture-of-experts (MoE) model, the largest open model ever released. T means trillion and B means billion. The model stores 2.78T parameters, but its router activates 104B for one token, so most of its specialist capacity stays out of that individual computation. It also has native vision, a 1M-token, or one-million-token, context and an Artificial Analysis (AA) ranking of#1 of 111 models on that index. The chapters that follow separate the release claim into its parts: what K3 stores, what it retrieves, what it routes, and what it learned to do over long tasks.

K2 → K3 (paper Table 1, verbatim):
  layers            61    →  93
  total params      1.04T →  2.78T
  active params     32.6B →  104.2B
  routed experts    384   →  896  (16 active, +2 shared)
  training context  128K  →  1M
  attention         61 MLA →  69 KDA + 24 Gated MLA
  scaling efficiency             ≈2.5× over K2

What is inside the model?

The parameter count alone does not explain the design. Kimi Delta Attention (KDA)is the delta-rule linear-attention mechanism in 69 of the 93 layers. It holds a compact running state, which keeps long-context memory from growing token by token. Attention Residuals (AttnRes) give a deep layer a weighted path back to earlier layer outputs, which reduces the need to pass every useful feature through every intervening layer.Stable LatentMoE routes each token to 16 of 896 specialists through a compressed latent representation, which makes a large expert pool more practical to train and serve. Per-Head Muon shapes optimization across attention heads, MoonViT-V2 brings image and video features into the shared backbone, and the four-stage context extension trains the model toward 1M tokens. Each part addresses a different bottleneck: sequence length, network depth, model width, visual input, or training stability.

The openness scoreboard