Automating Image Generation with Stable Diffusion
It’s hard to imagine that it was only January when OpenAI’s DALL-E-2 took twitter by storm by showing a new way to generate images from text. Initially there were only a very limited number invitations were available. Fast forward and now not only is getting access relatively easy (though no longer free), there’s an open source alternative: Stable Diffusion.
You can run Stable Diffusion on Google Colab for free or give it a try at Dream Studio. There are many other places to try too. Reddit has a list (of course they do). If you have a decent graphics card or a Mac with an M1 you can also run the model locally.
Now why would you want to go through the trouble of setting things up on your local machine when you can easily get access to a cloud instance? The answer is to be able to run experiments automatically. Let’s generate a travel guide to the United States!
We’ll start with a .csv with one line for each state with the name of the state and the top highlight — fun fact the top highlight was also picked by an AI (GPT 3). I had a little program generate the file, but you can easily create one by hand or from a spreadsheet.
We can now loop through the lines of the .csv using a simple bash script:
Running this script will create an image for each state named <statecode>.png. Depending on your hardware that could take a while, but that’s fine, just run it overnight.
Once it is done we can stitch the images together and turn the whole thing in a bit of an interactive app using a few lines of javascript. Give it a try!
All of this should be easy to adapt to generate other types of collections of images. Be sure to play around with the prompt generation too!
