shape keys with multires through sculpting

Sorry I’m not sure if I can post my opinion

I believe that the possibility of using
the sculpting mode to create shape key, even with multires modifier
would make it perfect for blender to
create complex and realistic facial animations

I once rigged the model, through the sculpting
I could create all those nuances of facial (with shape keys)
that changes in every area and continuously in the
time making sure that each photo is different.

sorry for the english…

there’s some inherent problems. but, I know that someone asked Ton on IRC about a similar thing.
They/He wanted to be able to keyframe sculpts for clay-mation type of animation with blender.

but you want to sculpt shapekeys? dunno if it’s possible.

I wonder how the sculpt data is stored, is it on a texture or something? like no sculpt is a 50/50/50 gray image, positive sculpt brush strokes is towards white 100/100/100 and negative is towards black 0/0/0.

maybe it could be posible to have several sculpt textures blend, like the shape keys for vertices.

I think it was possible to sculpt shapekeys with multires using some trick… Applying the multires modifier and using the hires meshes as shapekeys (join as shapes) works, but it’s not suitable for animation that involves armatures.

Multires data is definitely not stored as a B/W texture, as it can store 3-dimensional displacement. For this you’d need a vector displacement map and Blender doesn’t support those. You may be able to animate blending displacement textures with some fancy node setup though.

I didn’t mean the multires data, but the brush strokes in sculpt mode. how are they saved? that data should make sense to save as a bitmap of some kind. then it could be possible to animate a transition from one bitmap to another with nodes.

I think regular sculpt mode just modifies the mesh data directly. The brush itself is most likely a bitmap, but after a stroke all the changes are applied to the mesh in a “destructive fashion”. So if you are not using multires you can sculpt shape keys, as long as the topology/vertex count is exactly the same. But again, that’s no good when armatures are involved…


I do not know if it’s technically possible …
but think how nice it would sculpt all shapes
keys you want to quickly
on a model rigged with multires.
one thing that makes the computer graphics too brief
is for example the use of a limited number of shape
keys that are repeated.
in fact, in the real life any expression of a face is diffferent
and have infinite shades

i kinda see what your getting at my solution would be to somehow control pinching easier you know the upper lip, cheek bone, nose crease some are more defined than others especially when you get older not as wide a use as your idea but possible start i guess

Hi guys, I think that THIS is a pretty similar approach to what @marcolorenz says. It was a script (or addon… I don’t know), that allowed the animator to polish every pose of the character with the sculpt tools. Now it comes with the newer releases of Blender by default.

I’m just learning all this rigging and facial deformation stuff (with Stop Staring by Jason Osipa) so maybe I’m wrong, but it could be possible.
With Zbrush and (Maya, Max, XSI, etc) you can sculpt the blendshapes and then import them one by one, but it ends up being the same bunch of blendshapes with controllers 'cause the sculpt tool is an external app and doesn’t allow to make changes “on the fly”, and only works on static meshes… With Blender, the sculpting mode could be used not to create blendshapes, but to polish each keyframe and store it independently (as secondary shapekeys) from the main blendshapes. Now to have a pretty good result you would have to work with a really high resolution mesh, instead of using the multires modifier… I guess we will have to give it a try :stuck_out_tongue:

Wow, are people actually using multires with armatures?
I find it gives really sucky interpolation so will always bake to a map to use with SDS.

Since I started using 3delight I’m having to get more picky about bake quality, so my practice is changing but for shapekeys the ideal workflow really needs layers for sculpting.

Have a layer on the basecage for armature deformation of each pose and one at the highest level for the corrections…

Turn off the armatures deformation and bake out the correction.

Nb this means you must bake to retopologised mesh.

As blender doesn’t have layers then you have to duplicate the multires and bake instead… heavy dcenes!

Multires barely works for animation… it has bad interpolation. Ymmv, but baking is the way to go… even with the downside of normal direction only displacement and seam artifacts.

many thanks
I had already seen the video of anisculpt
In fact, if you want to use the sculpt mode
to create shape key with a model
rigged it can be done in Blender 2:59
But I wanted to do this on a model
rigged with multires because
would be how to animate the modeling
would be great for facial animation
credible and very rich in emotion.
I thought that if you want you could
apply the multires but it would
much better to keep the information
of multires.

reading this:

would someone know a way to convert sds+displace to multires? I have a basemesh+sudivision modifier + a displacement modifier (see blenderguru asterioid tutorial) and would like that data in a multiresolution modifier on the same basemesh for further tweaking with sculpting. Is that possible?

Sorry a little offtopic

remove the sds modifier, add a multires modifier (put it first in the stack)

subdivide the multires object to target level then apply teh displace modifier.

The displacement will be transferred to the multires object.

I’ve done high resolution shapekeys based on a displacement map.

What I did was to sculpt the base mesh at 2 million polygons or so. Once the base sculpt was done, I saved it as an external btx file.

Then, I copied the btx file and re-opened it into multires and started sculpting the muscle tension and facial wrinkles layer. I did this with Armature and Mesh deform enabled, and I posed the character and sculpted details for certain parts in certain poses.

Once I was done with that, I just saved the btx file. So, I ended up having 2 btx files, one with the base sculpt and another one with the extra details sculpt.

Then, I just opened two instances of my model and applied each of the btx file to them respectively. Then I baked the displacement for the details from the extra detailed high res model to the base sculpt high res model.
You may need some RAM to bake this High resolution meshes.

Well, after all that, I got my extra details displacement map!

Then, all I had to do was to add different displacement modifiers to the model (all using this same extra detail displacement map). Each modifier was limited to a certain area with a vertex group. So I had the disp modifier for the biceps, another one for the forehead, etc etc.

Then, I just added drivers to the influences of these modifiers based on the movements of certain bones.

Here’s the result.

@Michael W: thanx for that!
@jpbouza: impressive results!

Just out of curiosity has any one ever used shape keys on a mesh as dense as the ones in Avatar? Do you think Blender could handle a mesh that heavy, I think it had over 50000polys in the face alone. I know for that movie they didn’t use any displacement maps it was all handled with a dense mesh.

I think that in Avatar they used a kind of complex muscle and skin simulation system. A friend of mine once met a guy who worked as a rigger there and he said that the whole body was built up of muscles and bones, and that the skin was just simulated over the muscle structure, kind of a cloth simulation thing.

If the face was 50k polys then the whole body could have well been 150k or 200k… Blender might well handle that, but at 1 frame per second, hehe.

Anyway, I think that their techniques must have been very resource consuming, there are easier ways to go for obtaining the same results.
By the way, I didn’t like Avatar’s rigging results much, I kind of saw some flaws in the shoulders specially. I think some volumes were not preserved well enough, at least that’s the way I see it.
On the other hand, facial expressions were fantastic.

Concerning the technique I used in my video, it would be cool that Blender could have more than one Multires modifier and that it could handle an influence slider for each modifier. That would be cool, maybe Nicholas Bishop will do it some day, hehe.

I don’t about that because density seems to be the one constant I have noticed, I know the Benjamin button face was done differently from the avatar ones but the are both dense meshes far denser than Sintel or most of what I have seen people do in blender. Most blender faces I have seen are about as dense as the Gollum head.

I always follow this thread from cgtalk, Laa -yosh(Tamas Varga of Digic) has mentioned that most people have really gone up in density for realistic facial stuff. Judging by what those guys pulled of for the Assassins creed trailer it seems it pays of.

Mmm… yeah, there I see the Benjamin Button’s and Avatar’s topologies… Are you sure that those models aren’t subsurfed?

I can’t imaginw how someone could model in such density…

Anyway, this is an old and ugly example:

But what would be the difference between having such a dense base mesh, and having a rather simple mesh, add a subsurf modifier and then add other modifiers for details, such as muscles with shrinkwrap or wrap, and some displacements?

I think that there would be no difference…

As I see it, having a dense base mesh has the drawback of turning the mesh manipulation process far too complex.

In the example above, I had this crappy model I did with really simple topology, and then I added a sbusurf modifier to it. I created a muscle system beneath the character, and those objects affected the base mesh with shrinkwrap (which has a lot of drawbacks, the best way to go now is the wrap modifier I think). Those shrink wrap modifiers were put after the subsurf, so the topology for muscle deformation was much denser.

So, this way, you can do the rigging and everything with a simple mesh, able to be animated in real time, and then you have the muscles affect the mesh after the subsurf modifier, which results in much more polygons to get the desired detail.