I’ve been searching all through Blender forums, looking for a way to automate the lip sync process. It seems that lip sync must be applied manually, using shape keys.
If shape keys are all predefined, using a consistent naming convention, should it not be possible to automatically generate lip sync from an audio file, text file, or combination of the 2?
This would be really useful for making movies with Blender 3D. I would really appreciate it if one of you programming geniuses would take it on…
It is still a manual lipsync though isn’t it? One has to move individual phenomes into place to match the speech. However, there’s lots of software where one can just import the audio file, and the software reads it & processes the whole audio clip into phenomes. Like The Movies, or Moviestorm, etc.
Scripts already exist that analyze audio music files, converting them to IPO data that can be used to create synchronized animation - why not a script that analyzes a text file, or a combo of text & audio spoken words & process the whole thing?
I hope some python programmer(s) will take this up, it would be SO useful.
cheers
Enter your wav into the sequencer
set the shape keys for the first frame to Zero (so they show up in the action editor under shapekeys)
with your object/mesh still selected, run the script.
Select your .dat file from the papagayo program
hit the import button, then set each shape key for each phenomes (thats why its easier to name each shape key as A, E, O and such)
Hit GO and…you’ll see nothing…wow. yeah. Just move one frame and Oh! there we go now all the shape key animation set for you. A little adjusting and fine tuning will be needed yeah, but every program isn’t perfect, but yeah you don’t have to do it manually.
thumbs up Yeah it took me a bit too but it turns out it is a wonderful script. I wish it still were under development.
edit: Think I should set up a tutorial on how to do it with images and such on a site? I was thinking of making a site with tutorials for photoshop/blender/film type of thing
I’m actually working on a movie project with live action/3D character animation. A “who framed roger rabbit” type style movie. I’ve asked before if anyone wanted to help but didn’t get much of a response. Maybe after I post some footage more people would like to get on the bandwagon. once I get the lip-sync working with my latest character “Carl” for the movie I might show some 3D animation footage with him.
Looks cool, great shading. Is that a cigar in his mouth? I do a lot of machinima, and use Blender for a lot of modelling conversions & sfx but I’d like to do a full movie in Blender someday, I’ve got a long way to go in Blender, but I’ve done VOs & music for quite a few short machinima flicks if you need help in those areas.
I´m using papagayo for export the wav into dat, my character in blender have now 8 shapes for mouth the phonems that the Blendersynchro v 2.0 needs, but still it only recognize one mouth shape the first one after the basic…? Is this script working or not, the version that i´ve tried is 2.48a and 2.49b…
If is not working someone have a better solution for automate lipsync…?
