Xiaomi released MiMo-V2.5-Pro-UltraSpeed on June 8, a 1-trillion-parameter Mixture-of-Experts model that serves past 1000 tokens per second, near 1200 in demos, on a single standard 8-GPU commodity node. The headline is not the model size and not the throughput taken alone, it is the conjunction: trillion-parameter scale at speeds that previously required wafer-scale silicon (Cerebras) or custom SRAM-heavy architectures (Groq), now running on the kind of 8-GPU box a mid-sized lab or a well-funded startup actually owns. The checkpoint is open-sourced on Hugging Face as MiMo-V2.5-Pro-FP4-DFlash, select TileRT modules are on GitHub, and a limited API trial runs June 9 to 23 at roughly 3x the standard rate for roughly 10x the speed. The pricing line is the honest tell of where they think the value sits: you pay a premium for latency, not for intelligence.
The speed comes from three stacked techniques, and the engineering interest is in how they compose rather than any one of them alone. First, quantization that is selective rather than uniform: MXFP4 (4-bit) is applied only to the MoE experts, which hold most of the parameters and tolerate aggressive quantization best, while the rest of the network stays at FP8. Quantization-aware training preserves capability through the 4-bit squeeze instead of bolting quantization on after the fact. Second, DFlash speculative decoding operates at the block level: instead of drafting one token at a time, the draft model fills an entire block (capped at 8 tokens) in a single forward pass via masked parallel prediction, and the acceptance lengths reported are task-dependent, 6.30 for coding, 5.56 for math, 4.29 for agentic workloads, which tells you the method pays off most where the next several tokens are most predictable. Third, TileRT is the runtime that makes the hardware actually deliver: a persistent engine kernel that stays resident on the GPU and uses warp specialization to coordinate compute, data movement, and inter-GPU communication at microsecond latency, so the experts, the speculative drafts, and the cross-GPU traffic overlap instead of serializing. Quantization shrinks the bytes moved, speculation cuts the number of forward passes, and the persistent kernel removes the launch and scheduling overhead that would otherwise eat the gains. Each one is known in isolation. The result is the co-design.
The ecosystem reading is that this is a direct argument about where the inference-cost floor lives, aimed at the custom-silicon thesis. Cerebras and Groq built their case on the premise that commodity GPUs cannot hit the latency and throughput that frontier-scale serving needs, so you need custom hardware, which means you need their hardware, which means a hosted-API relationship and no portability. MiMo + TileRT is Xiaomi saying the commodity 8-GPU node can hit those numbers if you co-design the model, the decoding algorithm, and the runtime kernel together, and they shipped open weights and partial runtime code to make the claim checkable. If the claim holds under independent reproduction (the standard caveat, the published numbers are Xiaomi's own and the conditions like batch size and context length were not fully specified), the strategic consequence is that the moat custom-silicon vendors were building gets shallower, because the thing they sold as only-possible-on-our-hardware becomes possible on hardware you already have. That is the same open-substrate pressure we saw with Harness-1 in retrieval and Nemotron in speech, now applied to the inference-serving layer itself.
Monday morning, if you are serving large models and latency is your constraint: MiMo-V2.5-Pro-FP4-DFlash plus the open TileRT modules are worth a serious evaluation, but verify the throughput on your own node with your own batch and context settings before you believe the demo number, because the conditions were not fully published. If you are paying Cerebras or Groq specifically for speed that you were told commodity GPUs cannot reach: this release is the reason to re-run that buy-versus-build calculation this quarter, the 3x-cost-for-10x-speed trial pricing is a useful reference point for what the latency premium is actually worth. If you are designing inference infrastructure: the lesson is that the gains now come from co-designing the model, the decoder, and the kernel as one system, not from optimizing any layer alone, and the MXFP4-on-experts-only plus block-speculation plus persistent-kernel combination is the concrete pattern to study. And if you are tracking the open-versus-closed substrate thread, this is the inference layer's version of the same fight: the open option is closing the gap to the specialized closed option fast enough that the question is now when, not whether, the commodity node becomes good enough for frontier-scale serving.