This is a second version of the text-to-image generator with a controlnet (first controlnet version is here). The main differences: now allows uploading images, versus image URLs, and has more controls like width and height.
Params:
Prompt: This is the text prompt that you want the AI to generate an image based on.
Negative Prompt: This is the text prompt that you want the AI to avoid when generating the image. You can provide a list of negative prompts separated by commas.
CFG Scale: This parameter adjusts how much the output image should be influenced by the input prompt. A higher value for CFG Scale means that the output will be more closely aligned with the input, but may also be more distorted.
Width and Height: The resolution of the generated image.
Control Image: This is the image that the AI will use as a reference to generate the output image.
Preprocessor: This is a drop-down menu that allows you to select the image pre-processing method
Model: This is a drop-down menu that allows you to select the type of model that the AI will use to generate the image.
Weight: This is a slider that determines the importance of the guidance. A higher weight will result in a more faithful reproduction of the control image.
Guidance Start and Guidance End determine at what point in the ControlNet process the control image will be applied. Guidance Start sets the percentage of total steps from the beginning of the process that the ControlNet will start to apply the image, while Guidance End sets the percentage of total steps from the end of the process that the ControlNet will stop applying the image. Setting Guidance Start to 0 and Guidance End to 1 means that the ControlNet will apply the image to all steps in the process.
Models
- Canny edge detection: This model can be used to control the edges of the output image by providing an edge map of the desired shape or structure. For example, you can draw a sketch of a face and use it to guide the generation of a realistic face image.
- Depth estimation: This model can be used to control the depth or distance of the output image by providing a depth map of the desired scene or object. For example, you can use a depth map of a cityscape to generate different views of the city with varying perspective and focus.
- Pose estimation: This model can be used to control the pose or posture of the output image by providing a pose map of the desired body or limb positions. For example, you can use a pose map of a dancer to generate different images of people dancing in various styles and outfits.
- Segmentation: This model can be used to control the segmentation or labeling of the output image by providing a segmentation map of the desired regions or categories. For example, you can use a segmentation map of a landscape to generate different images of natural scenes with varying colors and textures.
- Scribble: This model can be used to control the scribble or drawing of the output image by providing a scribble map of the desired strokes or curves. For example, you can use a scribble map of a letter or symbol to generate different images of fonts or logos with varying shapes and styles.
- HED edge detection: This model can be used to control the edges of the output image by providing a soft edge map of the desired shape or structure. Unlike Canny edge detection, this model produces edges with varying thickness and intensity, which can capture more details and nuances. For example, you can use a soft edge map of a flower to generate different images of flowers with varying shapes and colors.
- M-LSD line detection: This model can be used to control the lines of the output image by providing a line map of the desired direction or orientation. This model can detect straight or curved lines in an image and use them to guide the generation process. For example, you can use a line map of a building to generate different images of buildings with varying architecture and design.
- Normal map: This model can be used to control the normal or surface direction of the output image by providing a normal map of the desired lighting or shading. This model can create realistic 3D effects in an image by using the normal map to adjust the brightness and contrast of different regions. For example, you can use a normal map of a sphere to generate different images of spheres with varying materials and reflections.
