Easiest way to get a face puppet complete with phoneme, emotion, eyebrow, etc poses?

I want to use it to create programmatic deep fakes (everything is on the up and up, i swear). I’ve completed the First Steps With 2.8 and the Donut tutorials which lets me move around blender a bit but the idea of rigging an entire face and syncing the lips to phonemes (adding controls for emotions, eyebrow raises, etc) seems daunting.

Specifically what I want is the ability to control the model from code. Using code I say:

  • look in this direction
  • blink
  • animate saying “It seems likely to me that UFOs come from extra-dimensional beings” at speed x.
  • look happy

…and it does that. As a non-animator/sculptor creating a rig like this would be difficult and time consuming. But I don’t see anything quite like what I’m looking that’s premade. Perhaps I could start with one of the characters on blenders website. For example, snow looks good but he doesn’t seem to have all the phonemes and he looks a tad cartoonish.

Can you advise on the quickest easiest way to get the type of puppet I am looking for. I will make it myself if I have to but perhaps it would be easier to buy a model?

You have many, many options out there for base characters…there are 2 that are the most common.

MakeHuman will allow you to make just about any character you can think of, and it is still open-source.

The other is Daz, if you want a higher quality as well as face morphs and phonemes built right in as well as animation, Daz characters are Fully rigged, including the face…along with shape keys and just about anything else you are looking for…

I prefer Daz, as it is far better IMHO…here is one I did a few years ago, exported from Daz and Rigified in Blender…


And, just a few of the Shapekeys I included in the import…

I’m going to add on to this… because it seems like the right direction to me.

Daz is free, and their Genesis base models are all free (I think?). So you can get in and start fooling around a bit, see if it might work for you. The Daz UI organization is a little convoluted, but nothing I haven’t been able to search my way to an answer online. I like The WP Guru’s content, not sure if he covers the basics.

To get into fully modifying Daz characters you will need to invest in some morphing controls. I would look at the Genesis 8 line (G8) because Daz has recently released Genesis 9 (G9), and there is a lot of G8 content and it is cheaper. That said, the G9 series streamlines to one figure which can do both male and female, so maybe you want to jump on board and move in the direction Daz is heading.

I find the Daz store pretty useless for searching, so finding things in it can be a slog. I’m posting what I think is my basic package: Daz3D’s face and body morphs, male and female, and then a third party set of extra head/face shapes.

G8 Female Head Morphs
G8 Female Body Morphs
G8 Male Head Morphs
G8 Male Body Morphs
200 Head and Face Morphs for G8 Male and Female

If time isn’t critical then I would bookmark things you’re interested in and check back, as the Daz store often runs sales, usually in the 15-25% range, but some big sales with stuff 70% off a couple times a year.

Daz has it’s own, free Daz to Blender script which you can install to Daz3D. I haven’t tried it so I can’t vouch for it. There is a free Blender addon called Diffeomorphic which migrates Daz characters to Blender. I haven’t used it but I understand it requires some user input, not as much as it used to, and there seems to be a fair amount of youtube videos covering the steps.

What I can speak to is a paid Blender addon called Rig-GNS. This is, dollar for dollar, easily the best money I’ve spent on third-party Blender addons. That said, Daz to Blender is pretty much the foundation of my professional workflow, so… know that I’m heavily biased. But Rig-GNS will bring a Daz Genesis (generation 3, 8, 8.1 and 9) over in 4 clicks, takes about a minute. It comes in fully rigged to Blender’s Rigify rig, with its materials properly set up for basic surface color. On the Daz side you have to export an FBX first, get a couple settings right, but that’s it. So on the strength of that alone this is a crucial addon for me. But the addon does more, including make it easy to create mouth controls for phenomes or any other facial shape, and also easily import animations to the character with a few clicks. What animations? Well, I’m thinking mostly of body stuff; Rokoko recently released a free online AI tool that will take video you shoot and translate it into 3D animation. Rokoko is one of the supported animation sources. I haven’t tried this out yet, but it seems to me like it should be a fairly straightforward pipeline from shooting a video/action on my phone to getting it into a custom character, made in Daz, operating in Blender. I’ve imported some animation from Mixamo, pretty cool, but haven’t worked with this or the facial morphs much yet. Here’s some videos on Rig-GNS.

Rig-GNS Import Daz Character
Rig-GNS Daz to Blender Rigging
Rig-GNS Lip Synch
Rig-GNS Rokoko AI Animations in Blender with Daz Figure

So, none of this might be as absolutely dead simple as you’re looking for, but this seems like an avenue that’s worth investigating for your needs.

1 Like