Using AI to generate concept art or reference images

I downloaded an off line version of the new VQGAN+CLIP type AI that generates images from text input.

it will generate thumbnail or approx 300x300 sized images that clear up to something useful in about 30 mins or less even on a GTX 1070.

Here is some examples i generated.

Text input = Tall old fisherman’s shack in a coastal bay with rough ocean waves and an old row boat.
Cliffside rocks over grown with ivy and moss and a sea mist rolling in.

Output =

Some song lyrics input “looking down from Ethereal Skies
Silent crystalline tears I cry
For all must say their last goodbye -
to Paradise”


Input text “imagining the unimaginable”
Output =

On a modern RTX card with tensor cores i think it might run at something like 300X the speed if it uses low precision compute.
Future improved versions might become really useful for generating larger and better images.

There is an online version up on google colab but its really a pain in the ass, it takes a long time to setup executing each codeblock one by one and for the non subscription users its limited to antiquated telsa K80 cards from the year 2014 and you can only use them for a few hrs a day until it blocks you…
Really not worth bothering with if you have your own PC.

Search for a program called clip app from here
a very simple app version that just works.

1 Like

I always wonder why humankind want’s to kill fantasy… take any text, get any image to any noun and glue it on a piece of paper… yes i’m meant the real ones… every kid does this in kindergarten… (simple) photo collage… (or maybe they glue smartphones together these days???)… Artist will tell you a story when they see their old sketches, what they thought while putting togeter a mood board and all this will go into the end result… But next grown up humans will press the button “Make Art” and call out: "Look mommy look! I have done art." … oh well, not mommy…

And this cooperation advertising text: “…using newest technology…future version will…infinity possibilities…” (give me all you money…)

(No offense intended.)

There are many people on the internet using this and claiming that they themselves are “making art”, and some of them probably actually believe it. Some make youtube instructional videos on how to press the play buttons on the VQGAN site and what artists names and styles to type into the text box in order to get the best results in the style of said artists.

And some youtubers think its also a good idea to sell the results as NFTs. As if its some work of their own that they conceived of and poured time into.

And yes the companies behind this tech will certainly try to sell it as a product once it becomes viable and they have enough compute power and algorithms to turn the text into large pieces of convincing artwork.

It’s the whole conundrum of AI, it will become better than us in every area and skill even art, if its allowed to continue learning.