Stable Diffusion

Official Stable Diffusion Download Page, AI Art Painting, and Image Creation Tools Website

Introduction

Stable Diffusion is an AI image generation app that can simulate and reconstruct almost any visual concept with simple text prompts. The app offers a web browser-based frontend interface, created with the Gradio module, allowing users to easily access Stable Diffusion's features in a low-code GUI. The Stable Diffusion Web UI provides various functions like txt2img, img2img, and inpaint, as well as upgraded features like model fusion improvements and image quality fixes. Users can adjust parameters to create different effects and engage in AI creation locally. Additionally, users can train their own models through the Stable Diffusion Web UI, offering multiple training methods to master relevant training techniques and create custom models.

Official Stable Diffusion Website:

https://beta.dreamstudio.ai/generate

https://stablediffusionweb.com/

Working Principle

Below is an overview of the working principle of Stable Diffusion.

Stable Diffusion is a versatile, multi-functional model. It can generate images from text (text2img). The image above shows an example from text input to image generation. Moreover, we can use Stable Diffusion to replace or alter images, which requires both text and image inputs.

Below is the internal structure of Stable Diffusion, understanding which can help us better comprehend its composition, the interaction between its components, and the meanings of various image generation options/parameters.

1. Composition of Stable Diffusion

Stable Diffusion is not a single model but a system composed of multiple parts and models.

Internally, we first see a text understanding component that converts text information into numeric representation to capture the text intent.

This section provides a general introduction to ML, with more details to follow. The text understanding component (text encoder) can be considered a special Transformer language model (to be precise, it's a CLIP model's text encoder). Inputting text into the Clip text encoder yields a feature list, with a vector feature for each word/token in the text.

The image generator then takes these text features as input and is composed of several parts.

Two steps of the image generator:

1. Image Information Creator: The Image Information Creator is a key component unique to Stable Diffusion and is the reason for its superior performance over other models.

The Image Information Creator runs multiple steps to generate image information. The step parameter in Stable Diffusion's interfaces and libraries is generally set to 50 or 100 by default.

The Image Information Creator operates entirely in the image information space (also known as the latent space), making Stable Diffusion faster than previous diffusion models that operated in the pixel space. Technically, the Image Information Creator consists of a UNet neural network and scheduling algorithm.

The term "diffusion" describes what happens in the Image Information Creator. Since the Image Information Creator processes information step by step, the image decoder can then produce high-quality images.