Most large language models generate text from left to right, predicting one token after another. The approach is simple and reliable, but inherently sequential: the next token cannot be produced until the previous one exists.
Google DeepMind’s experimental open model DiffusionGemma takes a different route. Instead of committing to one token at a time, it treats a block of text as a canvas and refines it through discrete diffusion. Up to 256 tokens are processed in parallel, allowing an answer to emerge through iterative denoising much like an image produced by a diffusion model.
From next-token prediction to a text canvas
An autoregressive model finalizes one new token at every decoding step. Even on a massively parallel GPU, this process repeatedly loads model weights for a small amount of work, making memory bandwidth a common bottleneck.
DiffusionGemma begins with a 256-token canvas. Its decoder applies bidirectional attention across all positions, while the sampler repeatedly predicts the canvas, keeps high-confidence tokens, and re-noises uncertain ones until the block stabilizes.
This changes generation in two important ways:
- Tokens within a canvas can use information from both directions, not only the text to their left.
- A token can be revised during later denoising steps instead of becoming permanent as soon as it is emitted.
For outputs longer than 256 tokens, the model still moves forward block by block. Once a canvas is complete, it is encoded into the KV cache before the next canvas is generated. The design is therefore best understood as block-autoregressive generation with diffusion inside each block.
Why parallel denoising can be faster
Processing a full block shifts decoding away from a memory-bound sequential workload and toward dense parallel computation—the kind of work GPUs handle well. Google reports more than 1,000 tokens per second on a single NVIDIA H100 and up to roughly four times the output speed of a comparable autoregressive Gemma model in suitable single-user, small-batch workloads.
DiffusionGemma is built on a Gemma 4 mixture-of-experts architecture. It has about 25.2 billion parameters in total while activating roughly 3.8 billion per step. Quantized versions can fit within the memory range of high-end consumer GPUs, making the model relevant to local inference as well as data-center experiments.
Bidirectional generation may matter more than the headline speed
The throughput figure attracts attention, but the ability to view and revise an entire block may be the more consequential feature.
It is naturally suited to non-linear tasks such as:
- Code infilling, where both sides of a missing section provide context;
- Inline editing, where a local change must preserve the global structure;
- Structured output, where brackets, tags, and schemas can be checked as a whole;
- Agent loops, which benefit from rapidly generated plans, tool arguments, and intermediate results;
- Local assistants, where a single user needs low latency rather than high aggregate throughput.
The model also supports text, image, and video input, a context window of up to 256K tokens, function calling, and configurable reasoning. Its open weights are available under Apache 2.0 through Hugging Face, Kaggle, and Vertex AI.
The tradeoff: this is not a universal Gemma 4 replacement
Parallel generation does not remove every compromise. Google’s model card shows DiffusionGemma trailing the comparable autoregressive Gemma 4 model on many knowledge, coding, mathematics, and vision benchmarks. Diffusion also requires multiple refinement steps, so realized speed depends on task difficulty, stopping rules, hardware, and batch size.
Its strongest advantage appears in low-latency, single-user workloads at low to medium batch sizes. High-throughput cloud services can batch many autoregressive requests efficiently, reducing the relative benefit of diffusion decoding. The “up to 4x” figure therefore describes particular workloads, not a guaranteed multiplier for every deployment.
The 256-token canvas should not be confused with finalizing 256 tokens in one forward pass. Adaptive generation commonly needs 12–16 denoising steps and may use up to 48 for harder tasks. Each pass progressively commits the positions in which the model has sufficient confidence.
A new decoding path for language models
DiffusionGemma matters as more than a faster Gemma variant. It demonstrates that language interfaces do not have to remain tied to visible token-by-token typing. A model that can draft in parallel, inspect a whole block, and revise its work may enable different approaches to code editing, structured generation, and real-time agents.
For now, it is best viewed as an experimental model for developers and researchers, not the new quality leader. In the longer term, it raises a useful question: do future language models still need to think like typewriters?
