Table 3 is vendor-run evaluation, not one common harness. Code agents use a one-million-token DeepSeek Harness setup; visual agents use a 512K Claude Code setup; AutomationBench and Agent's Last Exam use their official scaffolds.
Each row measures a configured agent, not a model in isolation. The scaffold, attempt count, and token budget change the result.
The wins
On DeepSWE v1.1, V4.1-Flash reports 74.2, against 74.0 for Claude Opus 5 and 73.0 for GPT-5.6 Sol. On Terminal-Bench 2.1 it reports 90.6, the top row in that table.
On AutomationBench it reports 54.8 against 50.3 for Opus 5. On Agent's Last Exam it reports 31.8 against 28.6. These are the workloads most aligned with long, repeated prompts and tool loops.
The base model shows a related result: MMLU-Pro 74.1, HumanEval 79.4, GSM8K 93.0, and BigCodeBench 60.6. The report says it matches V4-Pro-Base while using one third of the total parameters and one quarter of the activated ones.
The losses
Terminal-Bench 4.0 is the clear loss. V4.1-Flash reports 31.2, against 51.8 for Opus 5 and 39.9 for GPT-5.6 Sol. The report associates the gap with expert-level domain knowledge.
Humanity's Last Exam needs a careful reading. V4.1-Flash reports 36.8 on the full score and 39.1 on its text-only subset. Several comparison cells marked † are text-only subsets, so they are not interchangeable with the full score.
DeepSeek says the model completes more than 95% of real-world tasks. That claim is not public-benchmark evidence and deserves the same skepticism as any broad launch summary.
The bill, and why the cache is the product
One million input tokens with 80% cache hits and 200,000 output tokens costs about 15 cents at off-peak rates. The same task costs about 55 cents on DeepSeek-V4-Pro, $2.56 on GPT-5.6 Sol, and $6.40 on Opus 5.
Output dominates this example. Fresh input is $0.15 per million and cache hits are $0.003, a 50× discount. That price only holds if the service can keep the prefix resident.
The architecture and the price table point to the same bet: a small cache makes long-lived agent prefixes cheap to retain, not only cheap to read once.
Scoreboard, with the losses left in
Click a benchmark. Every number comes from the report's own tables.
Who this is for
This is aimed at agents that read more than they write: coding work across a large repo, screenshot-driven interface work, and document pipelines with a long working set.
For unfamiliar expert problems, the scorecard puts it behind the larger models. The right choice depends on whether your workload resembles the long-prefix agent loop above.