First Attempt at LipSync

I decided to go ahead and try some LipSync. Much harder than I thought. :yes:

Just incase you want to see the bones in action, SLOW motion:

Enjoy :slight_smile:

Downloading now.

Lipsync / animation should be a snap compared to modelling and rigging :slight_smile:

Just kidding :wink:



The sync looks good to me, albeit the voice track was spoken so quickly it was a bit hard to tell.

Now get his eyes, eyelids, head and the rest of his body moving :slight_smile:


Great first attempt! Did you make a shape key for each phenome? Actually, you mentioned bones…sorry…a pose for each? I would be very interested to see the .blend, since I a side thing I am investigating is re-use across meshes. Thanks.

high speaking rate complicates lipsync. in coaching some people on camera, i have to continually tell them to slow down. Slow down your speaking rate by about 2x. If you just said that to me that fast, i wouldnt understand it. in fact, i listened to it 3 times, and still dont quite understand what you are saying. THEN stretch out the lipsync, and then look at it in slo mo and you will be able to fine tune. Also, note that the lips have to move a frame or so earlier to form the air and make the sound, before we hear it.

No idea what he said, it sounded kinda like ‘aich dee a cavita, kaypee cluo’. So I really can’t tell if the lipsync is good or not, XP

OH, well then we have to get the “lil purpl martian dude witout da kool shades andat broken armitur” to comment. sorry. i thought it was King’s English.

whoa thats nice. i made a beaver lipsinc very easily with bones, but i don’t think the technique i used would be good on a human.
oh by the way, i think it says “what are you talking about, can’t you see i’m cool?”
yeah, make him say a little slower. otherwise, nice work. did you use shapekeys?

What are you guys talkin about? Can’t you see he’s cool??!! :wink:

Hmmm, Calvin. It’s your first and you used bones so it’s not that bad. I have no idea how hard it is to lip-sync with bones (though I’ve long-suspected it isn’t easy at all) so I can’t give any advice there. However, the method should be of no consequence, only the end result matters and there are a few issues…

He starts with his mouth open then it is closed just one or two frames later although he’s only going to say “what”, which isn’t a closed-lip shape so the lips move to that shape next. Start with a relaxed mouth, partially open and move straight to the first viseme from there. The current moves are distracting.

The next “major” problem is that his lips don’t visibly close on the “B” in “about” so it looks like he says “er-out”. The closed lip shapes are quite important visual cues so you might need to hold them an extra frame or two to let them register. To do this, you can hit the viseme early and leave a little late, sacrificing a bit of vowel sound if need be.

The tongue needs to do its thing on “Talkin”, “abouT”, “can’T” and “See” in particular.

His mouth also moves quite slowly despite some words being spoken quite fast. I’d say this “slow” effect is caused by the lips drifting from viseme to viseme. There need to be some held poses. The holds might only be a few frames, but they make a difference. I think the character doing the audio would have fast, jerky lip movements.

If you look at the Rocky thread in focused critiques, you’ll see comments about head and eye movements that apply here too. This is almost more important than the lip-sync.

On a “minor” note, I find the T-Pose distracting for lip-sync tests. If you aren’t ready to animate the arms, just drop them into a more comfortable position for now.

AndyD: Hey thanks for the comments, I’ll be sure to tweak it once I get home. It was not hard as in “hard to make the shapes” it was more of a timing. Just had to get the mouth to open and close at the right time. Will work on it.

noidtluom: I used bones for the mouth/eyebrows, but shapekeys for the face (Actually the shapekeys were too driven by bones.)

RogerWickes: I’ll post the blend when get home. (maybe I’ll tweak it first based on AndyD’s suggestions) :yes:

BlackBoe: Yeah, sorry… that was a bit too fast.

Mike_S: thanks.


Not this particular animation, but Calvin has posted the AJ rig, if you didn’t know, link to it and other rigs is in my Best of Blender thread (in my sig).


Here you go roger:

Little update:

MUCH better. Add some head movements and facial expressions and you’ll be there. For these movements and expressions, sit there and act it out yourself, in time with the audio. Even imitate the voice and BE the character. Lots of people use a mirror but I tend to just close my eyes and picture the expressions I’m making in my head. (Edit: as BlackBoe suggests below, a video cam of some sort can also be a big help too. Just don’t try to catch EVERY nuance - just the important stuff.)

Work through one or two elements at a time … head - eyebrows - cheeks - eyes and so on. My approach is similar to what Lisae suggested in the Rocky thread - start big with general movements and maybe some basic mouth movement (open/close) then work your way down to the details. Then I go back and tweak the head movements then tweak the details and this keeps going until I’m happy or I give up :slight_smile:

Ohhh… Heh, cool. Probably better than I’d do, but I’ve never tried. Anyway, I did notice something, when he says ‘what’re you talking about’ his eyebrows move for no apparent reason, they go up, then they twitch and go back down again, it doesn’t seem that natural. Like AndyD said, act it out in front of a mirror or picture it OR record yourself with a webcam or camcorder or some such. Also, when he says ‘cool’ the head jerk forward seems to have too smooth of a transition, I think it might be because the body and head move forward are at the same time, I think the head head should probably lead in the motion a couple frames before the body.

EDIT: Anyway, getting awesome. :]