MIMO-V2.6-PRO

A photo, a video clip, and a voice note enter this model through two encoders and land in the same token sequence as your text. The conversion rates are fixed and computable, so you can estimate a multimodal prompt before you send it: one token per 1024 pixels of image, 2 tokens per second of video, 6.25 tokens per second of audio.

Those rates are not cosmetic. They decide what a multimodal prompt costs, how much context a screenshot-heavy agent session eats, and why the audio path compresses as hard as it does.

How a photo becomes tokens

The vision encoder is MiMo-ViT: 28 layers, 681M parameters, a 1280-wide hidden state. It cuts the image into 16 by 16 pixel patches, then merges them in 2 by 2 groups, so each backbone token covers a 32 by 32 pixel block. A 2-megapixel photo becomes about 2,000 tokens. The processor caps an image at 8.4M pixels, which is 8,192 tokens, and floors it at 8,192 pixels, which is 8.

That merge ratio is the number to remember when you budget context. One full-resolution 4K screenshot is about 8,100 tokens, more than a page of dense text, and an agent that screenshots its work every turn will spend most of its window on images.

The attention inside MiMo-ViT is where the design gets specific. Instead of fixed, non-overlapping windows, which fragment a high-resolution image into isolated tiles, the local layers use sliding windows with a 64-token look to each side and an attention sink, so information crosses window boundaries over successive layers. Those layers alternate between reading the grid row-major and column-major, which spreads information along both axes. A global-attention layer at blocks 0, 9, 18, and 27 gathers the whole picture directly.

The recipe for training it is almost blunt. MiMo-ViT was pre-trained from scratch by pairing it with a small pre-trained language model and optimizing on multimodal understanding data with a cross-entropy objective alone, no contrastive losses, while keeping the language model trainable so the vision encoder received stable semantic gradients. More than 4T image tokens of that produced the encoder shipped here.

How sound becomes tokens

Audio takes two stages. The Audio Tokenizer, 308M parameters, reads log-mel spectrograms through a convolutional front end, a hybrid sliding-window and global transformer, and a second convolution that halves the rate to 25 frames per second. A 20-layer residual vector quantizer (RVQ) then writes each frame as 20 discrete tokens. This tokenizer follows the MiMo-Audio recipe and was trained on 20M hours of speech, music, and other sound.

Twenty tokens per frame at 25 frames per second is 500 tokens per second, and none of that should reach a language model. So the second stage, a 127M-parameter patch encoder trained jointly with the backbone, embeds each codebook separately, sums the 20 embeddings of a frame, and runs a small transformer over four consecutive frames at a time. The four outputs concatenate and project into a single backbone embedding. That is the 25 to 6.25 rate change: one token in the sequence per 160 milliseconds of sound.

What the input stack gives up

Everything is text out. This model can watch a screen recording and hear a meeting, but it cannot draw an image or speak, and the released checkpoints carry text generation only. The compression choices are real information losses, too: 6.25 tokens per second of audio is a coarse trace of a voice, and 2 tokens per second of video is one slide per half second of footage. Fine phonetic detail and fast motion are summarized, not preserved.

When you estimate a multimodal prompt, use the merged-grid rates above rather than the raw patch count. The merge is what makes a million-token window large enough for a session that watches and listens.