Diffusion Model
A diffusion model is an AI system that generates images, audio, or video by starting from random noise and gradually refining it, step by step, into a coherent output.
Key takeaways
- Diffusion models generate content by starting with random noise and gradually refining it into a coherent output.
- Training involves learning to reverse a process of progressively adding noise to real images.
- Diffusion models are the core technology behind most modern text-to-image AI generators.
- The same denoising approach has been extended to generate video and audio, not just images.
- Diffusion models largely replaced earlier approaches like generative adversarial networks for image generation.
What is a diffusion model?
A diffusion model generates content by reversing a process of adding noise. During training, the model is shown images that have progressively more random noise added to them, until they're pure static, and it learns to predict how to remove that noise step by step. Once trained, it can start from pure random noise and run that denoising process in reverse to generate an entirely new image.
How diffusion models generate images
Given a text prompt, a diffusion model begins with a canvas of random noise and repeatedly refines it over dozens of steps, using the prompt to guide each step toward an image that matches the description. Each step removes a bit more noise and adds a bit more coherent structure, until a finished image emerges.
What diffusion models are used for
Diffusion models are the technology behind most modern AI image generators, including tools that create art, product photography, and design mockups from text prompts. The same underlying approach has also been extended to generate short video clips and audio.
Diffusion models vs other generative approaches
Before diffusion models became dominant, image generation relied more heavily on architectures like generative adversarial networks, which pit two networks against each other. Diffusion models are generally more stable to train and tend to produce higher-quality, more diverse results, which is why they became the standard approach for text-to-image generation.
Frequently asked
- How do diffusion models work?
- They start from random noise and repeatedly remove a bit of that noise in steps guided by a prompt, gradually turning static into a coherent image.
- What is an example of a diffusion model?
- Most modern AI image generators, including tools that turn text prompts into pictures, are built on diffusion model technology.
- Are diffusion models only used for images?
- No. While they're best known for image generation, the same technique has been extended to generate audio and short video clips as well.