DiffusionGemma: what Google's open text diffusion model actually changes

DiffusionGemma is Google’s first open-weight text diffusion language model: a 26B mixture-of-experts with 3.8 billion active parameters, released under Apache 2.0. The headline claim is up to 4x faster text generation. A year ago Google demoed the same idea as Gemini Diffusion, but that stayed a waitlisted demo and the weights never shipped. This one you can download and run on your own machine, which is what I did. In this post I want to show you how diffusion generation differs from autoregressive decoding, what happens inside a single denoising step, what the speed costs in quality, and what I learned running it locally on Apple silicon.

One token at a time, no going back

Standard language models generate one token per forward pass. Every new token requires a pass over everything generated so far, so a 400-token answer takes 400 passes. And once a token is generated, the model cannot go back and fix an early mistake. It can only keep going and try to route around it. Generation is sequential, and it is permanent.

Google has been shipping open-weight Gemma models since February 2024, and every one of them has been autoregressive, including the Gemma 4 family from March 2026. Gemma 4’s multi-token prediction drafters attack the speed problem with speculative decoding, up to a 3x speedup, but the order stays left to right and committed tokens stay committed. Speculative decoding makes the sequential process faster. DiffusionGemma replaces the sequential process.

Diffusion flips the process

DiffusionGemma works the way image generators do. It starts from a canvas of placeholder noise and refines the whole thing in parallel, pass after pass. The full answer exists immediately as a rough draft, then sharpens everywhere at once. This also removes the no-going-back constraint: if a token from an early pass turns out to be wrong, a later pass can fix it.

Watching it run is the fastest way to build intuition. In a demo on my laptop, the model visibly went back and corrected earlier guesses as it refined, and the answer settled in about 24 steps, every position generated simultaneously. Twenty-four passes for a whole answer, instead of one pass per token. That is where the speed comes from.

Inside one denoising step

Each denoising step does four things.

1. The whole 256-token canvas goes through the network together, in a single pass. This is where the parallelism lives: one pass touches 256 positions instead of one.
2. For every position, the model scores its own uncertainty, the entropy of its prediction at that spot. In plain terms, it asks itself which guesses it is confident in.
3. The positions it is most sure about get accepted and locked, but only up to a budget. The budget stops it from overcommitting to a bad step.
4. Everything else is thrown back into noise, to be reconstructed next time.

Two refinements make this practical. The temperature cools linearly from 0.8 to 0.4 across steps, so the model explores early and commits late. And when the canvas stops changing, generation simply stops. The model card allows up to 48 denoising steps per block, but in my runs it usually settled in around 20.

That covers 256 tokens. For longer answers the model chains blocks. Once a 256-token block is fully denoised, it gets committed to the KV cache and frozen like ordinary context, and the next 256 positions are denoised conditioned on everything committed so far. The honest description is a hybrid: diffusion inside each block, autoregressive across blocks.

The network underneath

The speed also depends on each pass being cheap. The release is named 26B A4B; the model card puts it at 25.2B total parameters with 3.8B active per token. That is mixture-of-experts, which reduces to a simple idea: a big library of weights where each token only checks out a few books. The feed-forward layers are split into 128 experts, and each token gets routed to eight of them plus one shared expert that every token uses. Around that core sit 30 layers of sliding window attention with a 1,024-token window, a context length up to 256K tokens, and a small vision encoder of roughly 550M parameters riding along, so it accepts images as input. Cheap passes times few passes is the multiplication behind the headline number.

Speed, and what it costs

The official speed numbers: 1,000+ tokens per second on a single H100, 700+ on an RTX 5090, and up to 4x faster generation than comparable autoregressive decoding. The model card reports per-user speeds exceeding 1,100 tokens per second in low batch settings on an H100 at FP8, with the model locking 15 to 20 tokens per forward pass.

The cost shows up on the benchmarks against the autoregressive Gemma 4 26B A4B: MMLU Pro 77.6 versus 82.6, AIME 2026 69.1 versus 88.3, GPQA Diamond 73.2 versus 82.3. Google is direct about this: overall output quality is lower, and for applications that demand maximum quality they recommend standard Gemma 4. You are buying parallel decoding speed with benchmark accuracy.

Running it locally

What you need depends on the precision you pick. My rough breakdown from testing: the original BF16 weights need about 52 GB, which is A100 or H100 territory. FP8 cuts that to around 27 GB, still a 40 GB-class card. Quantized to 4-bit, Google’s own number is about 18 GB, which fits a high-end consumer GPU, and NVFP4 is supported natively on Blackwell. One caveat from my runs: memory grows with the context window, so budget beyond the weights for long prompts.

On my M2 Max with 96 GB of unified memory, running a 4-bit quantization through an early MLX port, I saw anywhere from 8 to 23 tokens per second depending on the prompt, and around 13 on a longer code generation task. Acceptable for a machine that is several generations old.

Serving support landed fast. Google’s developer guide lists vLLM, Hugging Face Transformers, SGLang, and MLX, and the Hugging Face page points to GGUF quantizations for llama.cpp, Ollama, and LM Studio. My recommendation: vLLM if you are serving customers and need batching, llama.cpp for local deployment.

Where diffusion pays off

The clearest use case comes from the ability to revise. My favorite demonstration is Sudoku. A small autoregressive model filling a grid one cell at a time tends to fail, because it cannot revise the cells it already committed. A fine-tuned diffusion model can complete the grid accurately, because parallel generation lets it go back and fix earlier positions. I showed one such community fine-tune in the video, and I am planning a follow-up on fine-tuning diffusion models for your own tasks.

Keep expectations calibrated. This is an experimental model. My Pokémon website test produced a usable page but nothing groundbreaking, which is what you would expect from 3.8B active parameters. The significance is not the output quality today. Gemini Diffusion proved the speed a year ago at 1,479 tokens per second and stayed closed. DiffusionGemma puts that research line in your hands, under a license that lets you build on it. The speed comes from denoising 256 positions per pass instead of one, the price is a few benchmark points, and now anyone can decide whether that trade is worth it.

Sources

• DiffusionGemma, Google DeepMind model page
• DiffusionGemma model card, Google AI for Developers
• DiffusionGemma: 4x faster text generation, the announcement post
• DiffusionGemma: the developer guide, serving and deployment details
• google/diffusiongemma-26B-A4B-it on Hugging Face
• Gemini Diffusion, the closed experimental precedent
• Accelerating Gemma 4: multi-token prediction drafters, the autoregressive speed play
• DeepSeek visual primitives: teaching models to reason with a cursor, another research note on this site