Hello all!
Wish you are well.
I have started an animation page basically, learned how to use procreate dreams to express my ideas via animation. I have million ideas but takes so long to do, which I don’t mind. But what particularly kills me is manually moving the lips/mouth per sound and even in a 90 second clip it takes me about 5-9 hours to do. This part is putting a damper on my motivation and I really am determined to keep it going.
I found blender and rhubarb auto lip sync to be a potential cure for my issue. But I’m not sure how to go about it , and because it’s so specific it’s hard to find YouTube videos on it.
All I want is for my characters mouths to move automatically with the sounds. I have a mouth shape for the some characters already ready.
If someone could help me overcome this, I will truly be forever grateful.
Thanks for your time reading this.
It’s not possible - unless you make 3D characters and render them out as an image sequence, then import that in Dreams. There’s no way to connect Dreams to Blender otherwise
Procreate is iPad only still, correct? In this sense, you can paint your characters in layers in Procreate but you’d have to export your layers as images and bring them into Blender on a desktop or laptop. If you can get COA Tools addon, you can do cutout animation with your images in Blender. Still a lot of work, but Ndee updated for 4.1 if I read that correctly.
Damn, chat gpt gave me hope that it’s possible, so I can’t upload the scene (video sequence) and leave the mouth blank and get rhubarb to do auto lip sync then ?
Yes mate that’s correct mate. Aahhhh so it’s not as easy as chat gpt makes out. I have the whole scenes ready just need the mouths to move automatically. I mean in some video styles the character doesn’t move at all. Essentially he’s standing beside a screen explaining the content displayed
Its good for stop-motion / cell style lipsync with 9 mouth shapes and no tweening between em. I also did some tutorials on how to use Rubarb with Blender. Not sure if the addon will still work as it uses the pose lib and tbats been replaced by the library.
Ah that’s some good work man. So everything on blender u must create essentially. For e.g if Homer Simpson didn’t have a mouth, and i put his image on blender and an audio over it , it would work? Unless I try and draw him from scratch?
OCA may be the solution of the interchange of choice in the future.
But you’ll notice that while someone is working on a photoshop exporter, nobody has even discussed the idea of bothering with procreate.
In theory you can export a PSD sequence to get your layered animation OUT of procreate.
I’m not sure if Krita supports opening a file sequence in PSD format but as far as I can tell THAT is your option unless you get a desktop build of procreate and find something more useful like an ORA file sequence exporter, which I KNOW for sure Krita should be able to open a sequence of.
IF YOU GET THAT FAR.
Then use a combination of OCA tools for Krita and Bluik for Blender
This will export your entire animation timeline into a directory hierarchy defined by the group folders of your document, which is SOOO much better than photoshop asset generator.
So essentially, all that work you may have done on different frames can potentially be saved using this method. But in addition it will trim all layers of unnecessary alpha, remember the positions of layers and with bluik it will keep those group layers organised as collections with the paint layers becoming objects. You can then go and add edge loops and geo and bind it armature etc like you would with COA, but you’d just do it manually.
I have not used your sound addon, but probably it’s a matter of using sound to drive the transformation of a bone along a certain axis path.
You CAN use COA tools, but keep it mind that it works within a very specific mode in Blender, while OCA tools is going to be a bit more flexible - particularly if this is just for animation.
PLUS COA tools will NOT allow you to export a timeline of animation where as OCA WILL.
However if you NEED something that can optimise your final animation for games, you WILL be better off relying on this updated branch of COA tools:
But you will still need to get your animation over to either photoshop or krita for option to work.
The ‘batch exporter’ addon that comes with Krita actually has a COA exporter built in.
To be honest, if I were you I’d considering swapping over to Krita.
But at least if you can get your animations over to Krita, you can in theory get them over to Blender.
