| description |
|---|
Explaining the ControlNet settings |
It is used to drag/drop the input image here. You can also click on the canvas and select a file using the file browser. The input image will be processed by the selected preprocessor in the Preprocessor dropdown menu. A control map will be created.
Create a new canvas with a white image instead of uploading a reference image. It is for creating a scribble directly.
Take a picture using your device's camera and use it as the input image. You will need to grant permission to your browser to access the camera.
Whether to enable ControlNet
For GPU with less than 8GB VRAM. It is an experimental feature. Check if you are out of GPU memory or want to increase the number of images processed.
Check this to enable a preview window next to the reference image. Use the explosion icon next to the Preprocessor dropdown menu to preview the effect of the preprocessor.
The preprocessor for preprocessing the input image, such as detecting edges, depth, and normal maps. None uses the input image as the control map.
ControlNet model to use. If you have selected a preprocessor, you would normally select the corresponding model. The ControlNet model is used together with the Stable Diffusion model selected at the top of AUTOMATIC1111 GUI.
{% hint style="info" %} Weight: How much emphasis to give the control map relative to the prompt. It is similar to keyword weight in the prompt but applies to the control map. {% endhint %}
It is used to control how much the control map is followed relative to the prompt. The lower the weight, the less ControlNet demands the image to follow the control map.
The ControlNet first applies 0 means the very first step.
The step ControlNet ends. 1 means the last step.
The initial steps set the global composition (The sampler removes the maximum amount of noise in each step, and it starts with a random tensor in latent space), the pose is set even if you only apply ControlNet to as few as 20% of the first sampling steps.
In conclusion, changing the ending ControlNet step has a smaller effect because the global composition is set in the beginning steps.
The ControlNet is applied to both conditioning and Un conditioning in a sampling step. This is the standard mode of operation.
The effect of ControlNet is gradually reducing over the instances of U-Net injection (There are 13 of them in one sampling step). The net effect is your prompt has more influence than the ControlNet.
Turn off ControlNet on Un conditioning. Effectively, the CFG scale also acts as a multiplier for the effect of the ControlNet.
Resize mode controls what to do when the size of the input image or control map is different from the size of the images to be generated.
Fits the image canvas to be within the control map. Crop the control map so that it is the same size as the canvas.
Fit the whole control map to the image canvas. Extend the control map with empty values so that it is the same size as the image canvas.
It supports using ControlNets multiple times to generate an image. (It helps us to pick different features from different pictures like CN0 provides pose, CN2 provides background, etc...)
{% hint style="info" %}
- Adjust ControlNet weights if one of them does not do its job.
- Pay attention to the resize mode if you have reference images of different sizes of the final image. {% endhint %}
{% embed url="https://stable-diffusion-art.com/controlnet/#Canny" %}