Google shipped native Multi-Token Prediction drafter support in LiteRT-LM today, the open-source production inference framework at github.com/google-ai-edge/LiteRT-LM. Yesterday's Gemma 4 12B release flagged MTP drafters as the latency-reducing piece baked into the model; the concrete number arrived this morning. Decode speedup is 1.6x on Gemma 4 E2B and 2.2x on E4B, with no claimed quality degradation. End-to-end, the LiteRT-LM runtime is reported at 1.8x to 3.7x faster than llama.cpp, MLX, Cactus, and ONNX on prefill plus decode for Gemma 4 variants. APIs expand beyond the existing Kotlin and C++ surfaces to add Swift and JavaScript, which puts the runtime in iOS, web, and Node.js paths without separate ports.
The mechanism is speculative decoding via a lightweight MTP drafter co-trained with the primary Gemma 4 model. Standard speculative decoding has been around in llama.cpp, vLLM, and the MLX ecosystem for two years, but the bottleneck has always been the data interplay between drafter and verifier: if the drafter runs on a different hardware IP than the primary model, the round trip eats the speedup. LiteRT-LM's specific engineering claim is memory locality enforcement, both the lightweight MTP drafter and the primary model execute on the same hardware IP (e.g., the GPU), which removes the cross-IP round trip that limits naive speculative implementations. That is the part worth pausing on for builders considering whether the 2.2x is reproducible on their stack: it depends on whether your inference framework respects the locality constraint or not. Numbers from Google's own benchmarks; independent reproduction on workloads matching yours is the standard next step before betting infrastructure on the figure.
Two ecosystem threads worth tracking. First, MTP-as-acceleration is now a concrete builder primitive rather than a research curiosity. The model ships with the drafter baked in (Gemma 4 mid-sized variants), and the inference framework respects the locality contract, and the speedup is published with comparisons to the actual frameworks builders use. That is the full stack you need for a speculative decoding feature to become defaults-on rather than expert-only. Other model families with MTP drafters (DeepSeek's MTP work, the speculative decoding research from Meta and others) now have a reference implementation pattern to follow. Second, the comparison cohort matters strategically. LiteRT-LM positions itself against llama.cpp (the open-source local inference default), MLX (the Apple Silicon default), Cactus (the mobile inference framework), and ONNX (the cross-platform serialization standard). Google is naming the incumbents and putting concrete numbers next to them, which signals LiteRT-LM is positioned as a contender for the default-local-inference slot rather than a research prototype.
Monday morning, if you are running Gemma 4 locally on llama.cpp, MLX, or ONNX today: pull LiteRT-LM and benchmark on your actual workload before deciding to switch. The 2.2x is the upper bound from Google; on consumer hardware the real-world number is more often in the 1.7x to 2.2x range, which is still meaningful but worth measuring on your context length and batch size. If you are shipping Gemma 4 in iOS or web apps, the new Swift and JavaScript APIs change what is feasible on those surfaces, treat as a fresh evaluation rather than relying on prior framework conclusions. If you are building inference infrastructure for any MTP-equipped model family, the locality enforcement pattern is the engineering lesson worth carrying over to other runtimes; co-located drafter and verifier on the same hardware IP is the part that makes speculative decoding fast in practice rather than just on paper. And as always with vendor benchmarks against open frameworks, the 1.8x to 3.7x comparison numbers are vendor-published; independent reproduction by the llama.cpp and MLX communities is the validation that matters.