Stable Diffusion Image Generation on a 4GB GPU and Windows
Stable diffusion is another OpenAI creation, which is also a free alternative to Dall-E, Craiyon, etc.
The beauty of SD is that it can run locally on your machine, and with a Graphics Card which has a small amount of VRAM. Supposedly it can be run on as little as 2GB, but I have an Nvidia A2000 with 4GB so we’ll use that.
Tip: Dont have a video card at all? Rent one using RunPod
With Stable Diffusion, the possibilities are endless as to what you can make.
The trick is how to prompt correctly, ie, ‘Girl with brown hair, holding camera’. Although this is a very short prompt.
Otherwise known as positive and negative prompts.
You need to really be specific, which means prompts can be 50-100 words in length.
Instructions thanks to Voldys SD guide.
I’ll post the steps below which I’ve used with great success.
Check your GPU has 4GB + using dxdiag.
You can click the Windows icon, type dxdiag & press enter
Check VRAM which will be shown in MB. You can see below that I have 3990MB of GPU VRAM.
Install Git for Windows
https://github.com/git-for-windows/git/releases/download/v2.37.3.windows.1/Git-2.37.3-64-bit.exe
When installing, make sure to check the box for Windows Explorer integration -> Git Bash
Use the defaults for all other options during the install.
Clone the WebUI repo to your desired location
Right-click anywhere and select ‘Git Bash here’
Copy the Stable Diffusion project to your local machine using Git Clone
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui
Download Stable Diffusion
Download the SD 1.5 Checkpoint from Huggingface
Link: https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned.ckpt
Rename the downloaded Checkpoint file tomodel.ckpt & move it to the \stable-diffusion-webui\models\Stable-diffusion\ directory.
Download Python
Download Python 3.10 from the link below
Link: https://www.python.org/ftp/python/3.10.6/python-3.10.6-amd64.exe
Ensure you select add to PATH when installing
Set GPU VRAM Requirements
We’re going to alter the SD configuration file so that it can run with GPUs with low VRAM.
Edit webui-user.bat with any text editor. This file is located in the base directory.
Locate the line COMMANDLINE_ARGS= and replace it with:
4GB GPUCOMMANDLINE_ARGS=-medvram
2GB GPU (Untested) COMMANDLINE_ARGS=-lowvram
Run Stable Diffusion
Launch webui-user.bat as a normal user, not as administrator.
Now wait patiently whilst it installs dependencies and does a first-time run.
It may seem “stuck” but it isn’t. It may take up to 10–15 minutes.
And you’re done!
How to Use Stable Diffusion
Launch webui-user.bat
After loading the model, it should give you a LAN address such as ‘127.0.0.1:7860’
To exit, close the CMD window
Enter the address into your browser to enter the GUI environment
Tip: Hover your mouse over UI elements for tooltips about what they do
Click Generate to begin
Whilst your Stable Diffusion is working away you’ll see activity in the command line. Make sure to keep this open. If you close it, SD will shut down
SD generated house
txt2img Good vs Bad
Negative and Positive prompts are the key to success with SD.
There are a myriad of other settings, but using these correctly will be an excellent starting point.
Bad Example
Let’s start with the bad example.
I’ve entered 1 girl as a positive prompt but no negative prompts.
We need to be more descriptive.
Good Example
I found these prompts on Reddit (I can't remember who, if you know please link me to their work so I can credit them! Sorry if you are the author!!)
Make your positive and negative prompts descriptive.
When an image is generated with a feature we don’t like, we add it into the negative prompt.
For example, in the above image, we can see some extra limbs in there. If we weren’t happy with our extra limbs there, we’d enter extra limbs as a negative prompt.
Pretty straightforward forward huh?
See Negative & Positive prompt examples
I’ve researched some negative and positive prompts which can be found:
