Control vertex color using spahekeys

Hello everyone,

Is it possible to control the vertex color (directly the color or just the intensity of the color) with shapekeys ?

For example I have the “mouth_happy” shapekey set to 0.
When I set it to 1, because it plays with the corners of the mouth, i want the vertex color in this region to become white or to have the intensity to 1.

Why ?
Because this system allows me to have only 2 normal maps and a better optimization in UE4. One standard and one for every expression deformations combined. So if the vertex color only appears when the shapekey is activated i can get better details without using a NM for each one of them.

Unfortunately, I don’t think a shapekey contains vertex colour data. I like your thinking - using a vertex group to fade between two normal maps - as the face distorts it fades between them.

There IS the possibility to have multiple vertex colour layers - In Blender you can fade between these with mix shaders activated by drivers linked you your shape keys… in unreal… I dunno. I don’t know if the FBX format can handle multiple vertex group layers for a start.

image

There is this addon


which creates a map of which bits of a mesh are compressed and which bits are
… but…
  1. I don’t know if it works with shapekeys
  2. I don’t know if it works with unreal

Also -

Welcome aboard! :smiley:

1 Like

Thank you :smiley:

And…

Thank you for your answer ahah.

That’s what I was afraid of. I could ask on the unreal’s forum if it would work but they aren’t as fast and friendly as here (when they answer)

What the add-on does is basically what I want to reproduce in ue4.

Maybe instead of vertex color I can use vertex weight, but it’s no more a Blender problem ^^
There is also the edges lenght solution but it seems too complicated for me ahah.

Thanks again :slight_smile:

Yeah, that could be a good solution.

BTW while shapekeys DO transfer - shapekey animation data does not. At least it doesn’t automatically. I’m sure there’s a way to transfer it.

Also - are you aware of Rhubarb lip sync?

There may be a way to import the rhubarb lipsync data into unreal and have it trigger shapekeys. Rhubarb its self exports a list of times in seconds with decimal places and letters for each key.

Actually animation data transfers as well.

I made a quick and simple rig with 4 bones controlling shapekeys (mouth corners and eyebrows going up and down).

Here is the result in UE4 :

HeadAnimation

But as you can see there are not so much details.

I never heard about this, is seems very good ! I will try it but I don’t know if it will fit my needs (it doesn’t use a face rig, more like a “menu rig” but the results seem really great)

For the face animation i was planning to use this method with a DIY mocap helmet (bike helmet and 2 pieces of wood to keep my smartphone always recording my face at the same distance.)

It seems that there are a lot more solutions that I thought :grin:

You can use Rhubarb with a face rig… you basically create a pose-lib pose for each mouthshape.

Rhubarb, however, is designed for stopmotion/cell/pixel-art style lipsinc (created for the game Thimbleweed Park)… so it works best for faces that will be 1 of 9 mouth shapes rather than being able to tween between them. It all depends on the style you’re going for. I would rhubarb would work better for a more cartoony look.

Face Motrack at the moment only works from the nose upwards - not the mouth.

There is also this option for lipsinc (very simple)

I think I have more tests to do with the facial rig :slight_smile:

For the details I will add more tris on the head for the main characters and it will solve my problem (not really but it’s the best solution available).

Thank you for your time and knowledge yogyog :kissing_heart: