KIMI K3

Reading the Scoreboard

chapter 6 of 6 · about 5 minutes

A benchmark is a fixed set of tasks with a stated setup and scoring rule. It tells us how a model did on that test. It cannot collapse every kind of work into one number. Moonshot's paper says Kimi K3 “consistently outperforms other open and proprietary models” in most categories, “while its overall performance still trails the most powerful proprietary models.” The bars below unpack that sentence without pretending that one bar settles the question. Start with the card title. It tells you the kind of work being measured. Then read the unit. A percentage is a result on that particular task collection, not a percentage of all coding or reasoning a model can do. A higher percentage is useful within that card, but 88 on one benchmark and 63 on another do not share a difficulty scale. Next, ask who ran the comparison and under what conditions. The paper's tables are Moonshot-reported results. Its Kimi K3 evaluations use maximum reasoning effort, and its coding evaluations use named agent setups such as Kimi Code, Claude Code, or Codex. For Terminal-Bench 2.1, the paper reports the best score across those setups. The paper also flags potential fallbacks in Claude Fable 5 results and potential cyberguards in GPT-5.6 Sol results. Those details do not erase a score. They tell us what had to be held fixed before comparing it. Artificial Analysis (AA) is shown separately because it is an outside index with its own suite and measurement method. Read each row as a result under its named benchmark, effort level, and source, then use several rows before deciding what kind of work the model appears to handle well.

Long-horizon coding, the SWE-Marathon row

SWE-Marathon % (paper §6, self-reported, max reasoning effort)

Kimi K3
63.7
Claude Opus 4.8
42
GPT-5.6 Sol
40
Claude Fable 5
39
GPT-5.5
35
GLM-5.2
14
  • Kimi K3: open weights, downloadable

Terminal-Bench 2.1, a close cluster

% (paper §6)

Kimi K3
88.8
Fable 5
88.3
Opus 4.8
88
GPT-5.5
84.6
GLM-5.2
83.4

    ProgramBench, a relative weakness in this table

    % (paper §6, 'trails the most powerful proprietary models', in their words)

    GPT-5.6 Sol
    77.8
    Fable 5
    77.6
    Opus 4.8
    76.8
    Kimi K3
    64.2
    • Kimi K3: 13+ points back on this one

    Independent check and the output-speed tradeoff

    Artificial Analysis (verified)

    AA Intelligence Index
    60
    Output speed (output tokens per second)
    39.5
    • AA Intelligence Index: #1 of 111 models tracked
    • Output speed (output tokens per second): #56 of 111, smart but slow
    The scoreboard covers capability. It does not answer the operating questions, so keep four other columns in view: input price, output price, cache reuse, and output speed. Moonshot quotes prices per million tokens rather than as a per-request fee. It lists $3/$15 per M tokens for uncached input and output, with $0.30/M for cached input. Input is the material you send in. Output is the material the model generates. The difference matters for an agent that reads a large project, calls a tool, then continues from the same history. A cache is a saved result from work the model has already done on a prefix. If the next request begins with that same prefix, the service can restore the saved state instead of processing the same input again. Moonshot reports cache hits above 90% on coding workloads. That is a statement about those workloads and their repeated prefixes, not a promise for every prompt. It explains why the cached-input price is lower, but it does not tell us the cost of a new long context or of a long generated answer. AA's Intelligence Index and output tokens per second answer different questions again. The index is a capability summary for AA's suite. Tokens per second is a delivery rate once the model begins responding. Neither one converts directly into the other, and neither one settles whether the price is appropriate for a specific job. The license is a final, separate condition. The weights are downloadable, but commercial deployment past Moonshot's scale thresholds needs terms. Scores, latency, price, cache behavior, and licensing are separate dials. A useful evaluation keeps them separate before combining them for a real workload.
    receipts, every claim in this chapter, checked 2026-08-31

    Chapter sequence complete

    Read another model, or revisit a chapter's receipts to compare its claims with the source material.