Facial rigging - What I've learned on Durian

[harkyman]

It's not repeitition of expression that leads to the mechanical look. People don't have a whole gigantic repository of expressions. They tend to make more or less the same "faces" time after time.

What leads to the mechanical look are the transitions between them. Different parts of the face lead and follow in different expressions. You can't just do a "smile" overall face expression. Well, you do it for artistic purposes, but then you break it up into several regions so you can key them at different rates.

Also, because momentum isn't so involved in these kinds of under-skin muscular contractions, you don't need nearly as much ease in and out as you would typically put into animation. More linear -- less bezier if you're really being picky.

[chipmasque]

Originally Posted by harkyman ...People don't have a whole gigantic repository of expressions. They tend to make more or less the same "faces" time after time...

I disagree here, I think there is a very great range, though subtle, in the expressive power of faces, much too great a range to be fully captured in a manageable set of shape keys. But the other points you make, particularly about transitions, are very good ones.

[mtracer]

Originally Posted by chipmasque I disagree here, I think there is a very great range, though subtle, in the expressive power of faces, much too great a range to be fully captured in a manageable set of shape keys. But the other points you make, particularly about transitions, are very good ones.

A nice, sensible, non-slamming reply. I applaud you for critiquing so honestly.

[mtracer]

@FeelGoodComics: I think the massdeformation should be done using entirely deformers. No actions, no shape keys. I think that's the beauty of this workflow, a free-form control setup with realistic shapekey-based deformations. Just a thought.

[MarkJoel60]

Originally Posted by chipmasque I disagree here, I think there is a very great range, though subtle, in the expressive power of faces, much too great a range to be fully captured in a manageable set of shape keys. But the other points you make, particularly about transitions, are very good ones.

Yes, exactly.

It isn't the quantity of expressions, it is the way the 3D animation program arrives at the position.

Lets say I have a mouth in the closed position, one in a open smile position, and key one on frame 1 and the other on frame 10 -- then let the Blender calculate frames 2 through 9 -- I have no control on how it arrives at the shape. It knows the start and it knows the end, and it will simply tween its way to fill up the rest of the frames.

But that's not how your muscles move. Your mouth edges will move first (maybe) and pull the skin out. Then the lips will follow. It is not a "round" movement. It has angles. Some muscles go before the others.

There is no way Blender can calculate those nuances. It simply keeps halving the problem until it is solved.

It is exactly why professional animators use FK for arm movement, not IK.

So, shape keys are good to get you to the right pose -- after the bones get you most of the way there. Because bones are controllable, you can control not only control the deformation, but the path you used to get there.

At least, that's the theory

[MarkJoel60]

Originally Posted by mtracer @FeelGoodComics: I think the massdeformation should be done using entirely deformers. No actions, no shape keys. I think that's the beauty of this workflow, a free-form control setup with realistic shapekey-based deformations. Just a thought.

I think this is impractical. Bones cannot get the nuances of the shape -- unless the mesh is highly stylized. If we are trying to get lifelike movement in a near human mesh, there will have to have corrective blendshapes somewhere.

Maybe a mesh deformer can get you there. But it is hard to imagine that a less dense proxy can get you all of the expressions. I don't see any way other than eventually correcting the poses with blendshapes / shapekeys.

[feelgoodcomics]

@everyone
I'd sure like to see some blends to demonstrate these concepts Nothing fancy needed, just something tangible to showcase the points you are making (feel free to use the jim carrey model if you like)

[chipmasque]

Originally Posted by feelgoodcomics ...feel free to use the jim carrey model if you like...

A generous offer, thanks, wish I had time to take you up on it.

But this also brings up another very important point -- facial rigs by & large are (and need to be) model-specific, made to articulate the character already established by the base model. Sure there are common factors that all the rigs will need to some degree, but in considering whether a rig design works well or not, it has to be considered whether it works well for the character it's used on. A caricature of Jim Carrey will require much greater general plasticity but less finesse of small detailed motion than, say, an Avatar cast member. And these are only two examples of a very broad range of characterization options open to animators.

I think it has to be acknowledged that any facial rig design will be heavily influenced by character design, and thus no one rig design is optimally suitable. At best, the fundamental structure of the rig can be a common framework for many character needs, but in the end, what that character has to do as an actor will determine the rig's final configuration.

[harkyman]

Not only is it influenced by character design, but by what the character will actually do. One of the things you use your storyboards for is to determine which animation controls and assets (like facial expressions) you need to add to your character.

Also, on the topic of a range of subtly different expressions, this holds as well. Let's say you're doing a three minute short (a LOT of work), or even less, a thirty second TV spot. How many different "smiles" do you need to make? How often is the character's face going to be "on camera"? Sure, you can attempt to create an end-all-be-all facial rig for everything you do, but with some foreknowledge of what your character needs to do, you can skip 90% of it.

As for timing of facial transitions, the simple and effective way to do this is the method I mentioned upthread. You sculpt (or Edit mode) a single overall expression. It's a nice way to do it from an artistic standpoint. Then, using the "vertex group" masking functions of shape keys, you breaks these full expressions out into separate shape keys per region. So you get one each for the different sides of the mouth. One for the brows. One for the cheeks. Etc. Then you rebuild the expression during animation by mixing the different region-based keys. It allows you to create different combinations and variations on the expression itself, as well as varying the timing.

An honest smile starts with the eyes. Lead by a few frames with the "eye smile" keys. A dishonest smile starts with the mouth, and includes less eye. It's also more symmetrical. So, pull the mouth up before the eyes.

Blender has the tools.

[Cessen]

Regarding the linearity of shape keys:
I really don't think that's as big a deal as people make it out to be. And when it is, it can be worked around.

IIRC Gollum's face was done using mostly shape keys.
This is also a favorite example of mine:
http://www.youtube.com/watch?v=VK2N3PvaFcs
In the comments, the artist states that it's (mostly) just careful construction of shape keys.
Also, the characters in Angela's demo reel are all shape-key based:
http://www.youtube.com/watch?v=1dU2YSKGKpU

Regarding mesh deform:
This isn't the catch-all rigging feature that a lot of people seem to think it is. It's very valuable, no doubt. But it comes with its own set of problems.

It's really useful when you're trying to deform an entire volume in a smooth and consistent way. But that same smoothness can really bite you in the ass if you're not careful, or if it's just ill-suited to the problem at hand.

Dynamic binding also has its own issues aside from that. Namely, deformation artifacts with large rotations. See the first shots in this video with Frank:
http://www.youtube.com/watch?v=nj0CV70hSfM
Note the squashed face. That's thanks to dynamic mdef. And note that it's not a bug, it's just a limitation of the technique.
We fought with this all throughout BBB, doing nasty things like counter-rotating the entire rig to avoid the artifacts. So I prefer to avoid it whenever possible.

Regarding the mdef+corrective shapes proposal:
My limited experience with rigging so far seems to suggest that layered or hierarchical rigging solutions, while tempting, often just result in uneccessary complexity. In many cases similar or identical results can be achieved with fewer layers and simpler techniques.

That's not to say that layered approaches are bad. They are good for solving some problems. But to me they're more something to be explored when simpler solutions don't meet your requirements (which can vary from rig to rig).

If a layered approach *is* needed, I think it's really important to understand/decide exactly which layers are responsibile for precisely which things. It's too easy to let it get wish-washy, with imprecisely defined responsibilities, and then you end up with a muddy mess (like the BBB rigs...).

As for myself, I think I'd like to explore some shape-key-based face rig solutions, but with some other methods brought in to help solve specific problems (like sticky lips, deformations from external forces, cartoony head stretching, etc.). I'm increasingly thinking this is the way to go for primary characters.

Still not sure about control schemes, though. For Durian we're going with a slider-based system, but where sliders only appear based on what parts of the face you have selected (I'll probably blog about it sometime this week or next). It works okay, and will get the job done. But it still feels clunky to me. It's at least a decent fall-back solution, though.

[mtracer]

What about using a single control for multiple keys in different directions? ;( It seems like an effective way to control lots of keys without sliders...

See ya,

MTracer.

[harkyman]

mtracer: That's been done. I know I've done it.

BTW, I tried to enact my two layer system with shape keys plus the Mesh Deform modifier and a high level cage. Unfortunately, the Surface binding method of the modifier doesn't work for this. I logged a bug, but was told that it was really only for flat stuff like clothes. Regular Volume based binding has its limitations too, which I learned in my last short project. So, scratch that idea.

[thedaemon]

Gollum's face was most if not all shape keys. Bay Raitt did a great job as did the rest of the crew on Gollum's face. The thing about the shape keys Bay did (inside of Mirai a 3d application that you may not be familiar with) is that you can edit the shape key at any time during an animation and it saves only the data for the vertex or verticies that you move. Say you rotate the jaw with a bone, then you sculpt the lips how you want them, save the shape key to the rotation of that bone. Voila! Now, another useful thing Mirai has is that you can "over" shape key things. What I mean is that you can continue to rotate that jaw bone and allow the shape key to push farther than you originally sculpted. This is very powerful to me and I've yet to find another 3d application with this easy way of working shape keys. :\

[ZombieJohn]

Originally Posted by thedaemon The thing about the shape keys Bay did (inside of Mirai a 3d application that you may not be familiar with) is that you can edit the shape key at any time during an animation and it saves only the data for the vertex or verticies that you move. Say you rotate the jaw with a bone, then you sculpt the lips how you want them, save the shape key to the rotation of that bone. Voila!

I think this would be amazing.

FWIW there's a somewhat similar script tool available for blender right now, but it's no-where near as... blended, I guess.

http://blenderartists.org/forum/showthread.php?t=179979

[mtracer]

This is what I mean by deformers in a rig. You can move the controls wherever you like, and the face wrinkles accordingly. This is not a complete rig. Only move the mouth corner controls and the jaw.

Have fun!

[Cessen]

Originally Posted by mtracer What about using a single control for multiple keys in different directions? ;( It seems like an effective way to control lots of keys without sliders...

That works well for cartoony characters that only make simpler, more graphical expressions. That's how I set up the BBB characters. But for characters that need to make more complex and nuanced faces, the rig really needs to account for the facial expressions resulting from opposing forces. For example, the muscles in the face that pull the mouth corner up and the muscles that pull it down are often both triggering at the same time, and this creates a unique and important expression. So you can't just put them on the same slider, because then you wouldn't be able to trigger them at the same time. Same goes for the eyebrows.

There are some cases where you can get away with lumping multiple things into one slider, though, and I am trying to do that where possible. A lot of it depends on how stylized the character is, and the intented style of their facial expressions.

Originally Posted by thedaemon Say you rotate the jaw with a bone, then you sculpt the lips how you want them, save the shape key to the rotation of that bone. Voila!

You mean like Pose Space Deformations?
http://citeseerx.ist.psu.edu/viewdoc...10.1.1.28.2112

That would definitely be cool to have.

Now, another useful thing Mirai has is that you can "over" shape key things. What I mean is that you can continue to rotate that jaw bone and allow the shape key to push farther than you originally sculpted.

Unless I'm misunderstanding you, I'm pretty sure most apps can do this. At the very least, Blender can. Just set the max limit for the shape key higher than 1.0.

Blender 2.5 is also capable of editing shape keys in-place at any weighting, even while mixed with other shape keys and deformations. It's really quite a joy to work with, and makes getting shape keys to play nicely with each other a lot easier.

[mtracer]

http://www.vimeo.com/9867714

That's the full rig I've made, and some basic pose tests.

[thedaemon]

Originally Posted by Cessen You mean like Pose Space Deformations? http://citeseerx.ist.psu.edu/viewdoc...10.1.1.28.2112 That would definitely be cool to have. Unless I'm misunderstanding you, I'm pretty sure most apps can do this. At the very least, Blender can. Just set the max limit for the shape key higher than 1.0. Blender 2.5 is also capable of editing shape keys in-place at any weighting, even while mixed with other shape keys and deformations. It's really quite a joy to work with, and makes getting shape keys to play nicely with each other a lot easier.

Pose Space Deformations sounds about right from the summary I read on that link. The only problem Mirai has(d) is when breaking down the pose channels and such. Of course that could be the auto-ik and pinning of joints that made it crazy. Which is another thing I wish to science Blender had. The joint pinning made animating require no rigging, besides creating bones and setting rotation limits if you wanted. Maybe I should do a demo video of it :\

I will have to admit I haven't even began to touch blender's shape keys at all, so I am ignorant about it.

[MarkJoel60]

Originally Posted by mtracer http://www.vimeo.com/9867714 That's the full rig I've made, and some basic pose tests.

This is really interesting... Do you think you could do another video showing how the rig is put together?

The concept of using a mesh deformer to control the deformations, and then using a rig to modify the mesh deformer is a little beyond me.

A "Take apart" vimeo would be really helpful... if you have time...

[MarkJoel60]

Originally Posted by ZombieJohn I think this would be amazing. FWIW there's a somewhat similar script tool available for blender right now, but it's no-where near as... blended, I guess. http://blenderartists.org/forum/showthread.php?t=179979

Well, that script is very helpful, no doubt... but it is only a mechanism to transfer a modified mesh back to the original mesh AS A SHAPEKEY. So, it could be used to help create the shapekeys, but if I understand what Mirai is doing -- this is really nothing like that...