Generative AI add-on - Text to Video, Image and Audio

Imagine a video editor, where you generate your footage with text. Could be cool, right?

Here’s an add-on which allows you to do that. Generate movie, image, audio, music or speech directly into the VSE timeline, or generate media for a full sequence of text strips.

Get it here:

Video generated with the add-on:


Star Wars music video done in the Blender VSE using the ModelScope add-on:

Describe the footage you need inside the VSE editor, hit generate and there you have it:

More features added:

Latent Space - a short film generated with AI

Here’s a video where I started out batch-rendered a lot of clips using the Blender add-on:, and arranged the clips into a story in the Blender VSE. I upscaled and interpolated the video in Davinci Resolve and added sounds from Freesound and music from Mubert. I had a lot of fun making this. Its wild Modelscope is only a few weeks old, I can’t wait to see how this will develop in the coming weeks and months.

In the latest update of this add-on, you can also add AI generated audio directly into the timeline, and auto-convert text strips into AI generated video and audio:

1 Like

Anime model card(no watermark) and model card selector added to the Text2Video/Generative AI Blender VSE add-on:


This is facinating and kind of scary.

1 Like

An audio-film done with this add-on using the Bark model card to produce the dialogue, chatGPT SW assistant to produce the screenplay and Stable Diffusion for the images:

#DeepFloyd added to an experimental branch of the Generative AI #Blender #VSE add-on: - Download:… - Select the model card. - Enter hugginface token. - VSE sidebar: select Image - Generate. - Wait: 16 GB dl. - Needs 16 GB of VRAM

1 Like

1 Like

New set of anime weights trained on 512 x 512 by Shadowman:

Infinite length for Bark text2voice, so make audio books in Blender:

I am all in on generative AI but these AI Videos area all terrible :expressionless:.
Hoping for a rapid improvement soon though

Animov 512x512. It’s just generated in Blender(12 frames at 8 fps) and exported without further processing:

1 Like

Generated with the above add-on:

That last video looks a cut above the rest of the results of this addon … im guessing you started with the paintings and the ai generated movement from them? I would be interested to put some drawings of my own into it and see what the result is.

It’s only text to video. This was the prompt used: “A painting: “Summer Evening at Skagen Beach – The Artist and his Wife” by P.S. Krøyer, slow motion”

1 Like

I thought copying the style of an existing artist with ai was no longer allowed … is it different if the artist is dead and the work out of copyright?