KIMI K3

Long-horizon agent work is more than producing one answer. The model has to reason, take an action, observe what happened, verify the result, and continue from the changed world. Reinforcement learning (RL) trains that behavior by letting a model attempt a task and assigning a reward to the resulting trajectory. One such attempt is a rollout. For a long task, a rollout includes the growing context, tool calls, tool outputs, and the state of the environment where those calls happen. Kimi K3's paper describes trajectories that can run through hundreds or thousands of tool calls and millions of accumulated context tokens. Its §5.3 infrastructure section explains how the system keeps those long attempts resumable rather than treating every training step as a fresh conversation. That distinction is the point of task persistence. A tool call can create a file, start a process, change a browser session, or expose a failed intermediate result. The next model step needs to see that changed state before it can choose a sensible next action. Training only on a final answer would discard the decisions, observations, retries, and verification that made the answer possible. Long-horizon RL keeps those steps in one trajectory so the reward can evaluate the completed environment state, not only the model's text about what it claims to have done.

What has to persist during a long rollout?

• Context, a single rollout can stretch to 1M, or one million, tokens. A token is a small unit of the model's input or output, so this is a record large enough that reprocessing it at every pause is costly. Context parallelism splits the sequence work across compute workers, and an external key-value (KV) cache keeps a reusable processed prefix available for resumption. The cache holds the internal attention state produced while reading a prefix. K3 keeps active decoding blocks on the graphics processing unit, then writes reusable idle prefixes to an external cache pool in central processing unit memory when they are evicted. On the next reuse, it can prefetch that prefix instead of rebuilding the same long history from the beginning. That turns a pause from a full restart into a continuation.
• State, the agent can have files, processes, and browser sessions. A sandbox is the isolated computer environment that holds those things. Saving only the text context is not enough if the filesystem or running process has changed, so the sandbox needs checkpoints and a way to resume at the same point after a pause or worker interruption. K3 uses a micro virtual machine sandbox called AgentENV for this kind of work. Its checkpoints save only memory pages changed since the last checkpoint, which the paper reports can reach 133 milliseconds for checkpointing and 49 milliseconds for resuming. A paused sandbox can release its memory and central processing unit resources while it waits for model output. That is what persistence means here: the task's world does not have to be recreated every time the model or trainer switches to another rollout.
• Tasks, the model needs long problems whose outcome can be checked independently. In §4.2, Moonshot describes knowledge-graph-guided synthesis, verifiable agentic environments, kernel-optimization tasks, and web-development tasks. Those environments turn a final file, answer, or program state into a reward signal instead of relying only on the model's own claim that it finished. In Autonomous Execution Tasks, the paper says an independent verifier evaluates the final environment state. That changes the learning problem. The model must plan, choose tools, recover from errors, and stop at the right time because a self-reported success message has no value by itself. Press run the month below to watch a compact example of context and sandbox state moving forward together.

The results give these systems a reason to matter, but they do not prove that one infrastructure choice caused one score. This chapter's long-horizon callouts are SWE-Marathon 63.7 versus Opus 4.8's 42.0, Terminal-Bench 88.8, and a Vending-Bench-style economy comparison at 1736 versus GPT-5.6 Sol's 1747. Read them as outcome measurements alongside the training design. A score measures the final task result under a given evaluation setup. It does not expose every internal cause of the result. It does show why task persistence is part of the story: a model cannot learn to carry a task across many actions unless the rollout, its context, and its world can remain available long enough to finish the attempt and receive a result from an independent checker.