Model architectures, explained slowly
Dims & Sums
We read the technical reports, draw the mechanisms, and work through the arithmetic. Then you can move the controls and see what changes.
GLM-5.3-Flash
Qwen3.8-Flash-Next
Kimi K3
DeepSeek-V4On the table today
model 001: why a 320B model can charge $0.15 per million input tokens
GLM-5.3-FLASH
GLM-5.3-Flash pairs 320B total parameters with 18B active parameters, mixes two cheaper attention paths, and ships native FP8 weights under the MIT license. Read the chapters in order, then use the controls to test the tradeoffs yourself.
read chapter 1:Meet the Flash
Other models on the table
each one makes a different tradeoff between capability, cost, and context
QWEN3.8-FLASH-NEXT
Qwen3.8-Flash-Next relocates 51B n-gram parameters to host memory, combines Gated DeltaNet with Qwen Sparse Attention, and evaluates each design change against quality, training cost, and stability. The reported result targets 397B-parameter flagship quality with roughly one ninth of its training compute.
start,The 1/9 Recipe
KIMI K3
Kimi K3 scales Kimi Delta Attention, Attention Residuals, and LatentMoE to 2.8T total parameters. The technical report describes about 2.5 times more capability per unit of compute than K2; the chapters separate that claim into mechanisms, measurements, and open questions.
start,The 2.8T Statement
DEEPSEEK-V4
DeepSeek-V4 applies two compression paths before sparse attention spends the expensive compute. The reported architecture reduces its key-value cache to about 2% of a classic attention baseline at one million tokens. These chapters trace the compression steps, their quality constraints, and the cost model they make possible.
start,The Price of Intelligence