Imagine a video editor, where you generate your footage with text. Could be cool, right?
Here’s an add-on which allows you to do that. Generate movie, image, audio, music or speech directly into the VSE timeline, or generate media for a full sequence of text strips.
Get it here: https://github.com/tin2tin/Pallaidium
Star Wars music video done in the Blender VSE using the ModelScope add-on:
Describe the footage you need inside the VSE editor, hit generate and there you have it:
More features added:
Latent Space - a short film generated with AI
Here’s a video where I started out batch-rendered a lot of clips using the Blender add-on: https://github.com/tin2tin/text_to_video, and arranged the clips into a story in the Blender VSE. I upscaled and interpolated the video in Davinci Resolve and added sounds from Freesound and music from Mubert. I had a lot of fun making this. Its wild Modelscope is only a few weeks old, I can’t wait to see how this will develop in the coming weeks and months.
In the latest update of this add-on, you can also add AI generated audio directly into the timeline, and auto-convert text strips into AI generated video and audio:
Anime model card(no watermark) and model card selector added to the Text2Video/Generative AI Blender VSE add-on:
This is facinating and kind of scary.
An audio-film done with this add-on using the Bark model card to produce the dialogue, chatGPT SW assistant to produce the screenplay and Stable Diffusion for the images:
#DeepFloyd added to an experimental branch of the Generative AI #Blender #VSE add-on: - Download: https://github.com/tin2tin/Generative_AI/tree/DeepFloyd… - Select the model card. - Enter hugginface token. - VSE sidebar: select Image - Generate. - Wait: 16 GB dl. - Needs 16 GB of VRAM
New set of anime weights trained on 512 x 512 by Shadowman:
https://twitter.com/tintwotin/status/1654732926032945155
Infinite length for Bark text2voice, so make audio books in Blender:
https://twitter.com/tintwotin/status/1654408121823948801
I am all in on generative AI but these AI Videos area all terrible 
.
Hoping for a rapid improvement soon though
Animov 512x512. It’s just generated in Blender(12 frames at 8 fps) and exported without further processing:
Generated with the above add-on:
That last video looks a cut above the rest of the results of this addon … im guessing you started with the paintings and the ai generated movement from them? I would be interested to put some drawings of my own into it and see what the result is.
It’s only text to video. This was the prompt used: “A painting: “Summer Evening at Skagen Beach – The Artist and his Wife” by P.S. Krøyer, slow motion”
I thought copying the style of an existing artist with ai was no longer allowed … is it different if the artist is dead and the work out of copyright?
