====== Stable Diffusion ======

Stable Diffusion is an AI model for text to image generation. There are various GUIs available to tweak settings, add specific styles like Disney or photorealistic (Lora models) and help adjusting existing images via picture to picture or selection infill.

The guides below focus on the [[https://github.com/AUTOMATIC1111/stable-diffusion-webui/|Automatic1111  Stable Diffusion WebUI]]. 
===== Linux Docker install =====

This docker requires docker as well as the GPU drivers installed on the host system.

https://github.com/AbdBarho/stable-diffusion-webui-docker

<code>
git clone https://github.com/AbdBarho/stable-diffusion-webui-docker
cd stable-diffusion-webui-docker
vi docker-compose.yaml
#adjust name to stable-diffusion if desired
#adjust data and output dir location to /opt/stable-diffusion/data and output respectively if desired
mkdir -p /opt/stable-diffusion/{data,output}

#Build the ai and download the models.
#Note, first start will take 15-60 minutes to download models to data folder as cache depending on internet connection
#size approximately 10GB

docker compose --profile download up --build

# if download errors occur, just repeat the command.

# build the desired interface:
# docker compose --profile [ui] up --build
# where [ui] is one of: invoke | auto | auto-cpu | comfy | comfy-cpu

# use auto-cpu or comfy-cpu for cpu only interface

#docker compose --profile auto-cpu up --build
docker compose --profile auto up --build

# later:
docker compose --profile auto up -d

</code>
access the app on:

http://localhost:7860/

===== Windows install for AMD GPU =====

Windows AMD AUTOMATIC1111 stable diffusion webui using Microsoft DirectML for GPU acceleration:

  - Download and run installers for [[https://www.python.org/ftp/python/3.10.6/python-3.10.6-amd64.exe|Python 3.10.6]] (ticking Add to PATH) and [[https://git-scm.com/download/win|git]]
  - Clone the fork as AMD is not officially supported\\ Paste into command prompt in a directory where it should live\\ <code>git clone https://github.com/lshqqytiger/stable-diffusion-webui-directml
cd stable-diffusion-webui-directml
git submodule init
git submodule update
venv\Scripts\python.exe -m pip install --upgrade pip
venv\Scripts\pip.exe pip install -r requirements.txt
venv\Scripts\pip.exe install torch-directml
</code>
  - Right-click webui-user.bat and adjust the command line args to\\ <code>COMMANDLINE_ARGS=--use-directml --skip-torch-cuda-test</code>
  - If you have only 4-6gb vram, try adding these additional flags to webui-user.bat's command line argslike so:\\ <code>--opt-sub-quad-attention --lowvram --disable-nan-check</code>
  - Double-click webui-user.bat in that directory, this will download ~2.7GB torch and other python modules, followed by the 4GB Stable-Diffusion 1.5 pruned model.
  - If it looks like it is stuck when installing or running, press enter in the terminal and it should continue.
  - once done, a browser opens to [[http://localhost:7860/]]

Check/Test if GPU is used in Task Manager's GPU tab. Dedicated GPU Memory should show large usage for the loaded Model and when generating a test image in the web interface, the GPU usage should ramp up like here:\\
{{:ai:pasted:20240504-020430.png}}

===== Performance =====

Approx times to generate 512x512 default settings images:
^CPU/GPU^Model^Time^
|CPU|Intel 8th gen i5 (Intel NUC)|5-15 minutes|
|GPU|AMD Radeon 780M|1.5 minutes|
|GPU|AMD Radeon 7900XTX|21.7 seconds|
|GPU|Nvidia RTX 4080|1.2 seconds|

===== Gallery Addon =====

As an extension for SD-webui:
Open the Extensions tab in SD-webui.
  - Select the Install from URL option.
  - Enter https://github.com/zanllp/sd-webui-infinite-image-browsing
  - Click on the Install button.
  - Wait for the installation to complete and click on Apply and restart UI. 

{{:ai:pasted:20240518-142918.png?500}}

===== Cozy Nest UI Addon =====

As an extension for SD-webui:
Open the Extensions tab in SD-webui.
  - Select the Install from URL option.
  - Enter https://github.com/Nevysha/Cozy-Nest
  - Click on the Install button.
  - Wait for the installation to complete and click on Apply and restart UI. 

{{:ai:pasted:20240518-142848.png?500}}
===== general advice =====

[[https://www.reddit.com/r/StableDiffusion/comments/x41n87/how_to_get_images_that_dont_suck_a/|Some general Stable Diffusion prompt advice]]

[[https://machinelearningmastery.com/more-prompting-techniques-for-stable-diffusion/|More promptng techniques]]

[[https://learnopencv.com/sdxl-inpainting/|Inpainting explanation]] 

====== Models / Settings ======

LoRA models/styles/tools as well as base models can be searched and downloaded from [[https://civitai.com/models|CivitAI]]
===== Realistic Vision + Lora =====

Description for [[https://www.reddit.com/r/StableDiffusion/comments/11u2p0u/lazy_guide_to_photorealistic_images/|photorealistic images]]:

  - Download model [[https://civitai.com/api/download/models/130072?type=Model&format=SafeTensor&size=pruned&fp=fp16|Realistic Vision V6.0 B1 Base Model]] from https://civitai.com/models/4201/realistic-vision-v11 and save it into the ''models/stable-diffusion'' directory
  - create ''models/Lora'' directory
  - download [[https://civitai.com/api/download/models/217538|LORA Style hyperrealism art]] and save to ''models/Lora'' directory
  - download [[https://civitai.com/api/download/models/16576?type=Model&format=SafeTensor&size=full&fp=fp16|LORA epi_noiseoffset]] and save to ''models/Lora'' directory
  - in web interface go to extensions -> install from url and paste to install:\\ <code>https://github.com/mcmonkeyprojects/sd-dynamic-thresholding
https://github.com/opparco/stable-diffusion-webui-composable-lora</code>
  - then go to installed tab and click apply and quit
  - restart stable diffusion web ui
  - go to Lora tab, configure entries (which will create json files, then the entries disappear)

Base settings:
<code>
set sampling steps to 20
set sampling method to DPM++ SDE Karras
set width to 768 and height to 512
set CFG scale to 6
set seed to -1
enable Dynamic Thresholding (CFG Scale Fix)
enable all composable Lora
</code>


===== Stable Diffusion SDXL Model =====
<code>
cd /opt/stable-diffusion/data/models/Stable-diffusion
wget https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/raw/main/sd_xl_base_1.0_0.9vae.safetensors
</code>

===== Stable Diffusion XL Turbo model =====
This is for general objects/scenes/animals/etc, faster than SDXL

download [[https://huggingface.co/stabilityai/sdxl-turbo/resolve/main/sd_xl_turbo_1.0_fp16.safetensors|SDXL Turbo]] from https://huggingface.co/stabilityai/sdxl-turbo and copy into ''Models/Stable-diffusion'' directory.

===== Text LoRA =====

https://civitai.com/models/176555?modelVersionId=214296 

Other Text LoRAs:
https://civitai.com/tag/text
===== Image Style/Prompt libraries =====

[[https://lexica.art/|Lexica]]
[[https://prompthero.com/|PromptHero]]

===== Some test image prompts =====

<code>
3d, 8k, masterpiece, (best quality), Japanese garden, waterfall, cherry blossoms in full bloom, conifer, koi pond, (pagoda:1.1), stone lantern, water basin, water reflections, winding path, (night sky:1.:1), hokusai inspiration, ultra-realistic, pastel color scheme, soft lighting, golden hour, tranquil atmosphere, landscape orientation

EasyNegative, (worst quality:1.2), (low quality:1.2), (lowres:1.1), (monochrome:1.1), (greyscape), multiple views, comic, sketch, watermark

25 steps, 2m karras, 1024x768 seed 1192013237
</code>

<code>
greg rutkowski, highly detailed, dark, surreal scary swamp, terrifying, horror, poorly lit, trending on artstation, incredible composition, masterpiece

with settings:
 LMS Karras
 50 steps
 CFG scale 8
 random seed
</code> 

Test. use in txt2img the following prompt:
<code>RAW photo, close up photo of face, woman with dark hair in rain at night, blue eyes, detailed eyes, dark skin, (high detailed skin:1.2), 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3   <lora:lora_contrast_fix:1>  <lora:lora-style-hyperrealism-art:1> </code>
and as negative prompt below:
<code>(deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime:1.4), text, close up, cropped, out of frame, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid, mutilated, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck </code>
then click generate as often as you like

{{:ai:pasted:20240427-095213.png}}

{{:ai:pasted:20240427-095226.png}}

====== Update docker ======

https://github.com/AbdBarho/stable-diffusion-webui-docker/issues/683

If you want a more up to date version of A1111 here are the steps:
<code>
docker exec -it webui-docker-auto-1 /bin/bash
git pull
pip install -r requirements.txt
#if needed
cd repositories
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui-assets.git
exit

docker stop webui-docker-auto-1
docker start webui-docker-auto-1
</code>
If it fails for any reason just rebuild the container using the standard process of docker compose --profile auto up --build