Facial Animator uses a sophisticated neural network-based Lip-Sync engine, to enliven characters with realistic expressions to accompany speech, created directly from a sound file by generating human facial muscle movements for the entire human face.
Facial Animator can use any model of a human head, match to a real human face using the image editor using two photographs of an actual human face, and convert it into a fully textured, seamless 3D model.
[RIGHT] Modify shape of head in Image [/RIGHT]
"
How difficult is it to create code that can map morph the face mesh to match two pictures?
Will we see this in Blender within the year? I kow they are working on the facial things now (not the sound matching part though).
i think its called Blender LipSynchro, and there is some choice of two apps that convert text/sound to an action set, which you feed into LipSynchro to produce the shape keys.
as far as the UV Mapping and texturing thing, I don’t see how we could do an automatic edge-detect on the images to match/merge the, for example, nose UV to the spot on the image that is the nose, and upper lip, etc, based on ANY mesh model. We do have UV Project, which would do most of it, which would create two UV Textures. But one would have to be set to take precedence over the other (alpha?) so that UVFrontal would alpha to 0 as it got to the side of the head, and vice-versa for UVSide.
well, it’s not tooooooo hard to do, just time consuming to in side view, enter Face Select, select the front faces. Switch to front view, Unwrap->Project. In the UV window, image->Open->MyFront.jpg. Then tweak.
Repeat for side. Now you have two UV Textures. Go into paint mode and using the Alpha Erase, erase the overlap.
maybe 30 minutes total, including picture taking time.
LMPO! No not that way, silly papa. The mesh also would have to be changed/tweaked to fit the picture.
Case in point, unwrap monkey and apply a real picture of monkey. The model monkey’s mouth and nose are goofy and show through the image, you know, depth.
That was my first unwrap, And surprise to see it worked, but looked really, weird. I had to make those faces flat.
I don’t think they are morphing the geometry per se … I can’t tell from the teeny picture … but it is probably some thing more like the UV projection in Blender … But did you download the test animation (jolie8) on the left side bar ? … If that’s the result errrrrrrrrrrrrrr … no thanks .
no, you dont change the mesh; you stretch the image to fit the mesh. sorta like sticking your face into a piece of saran wrap.
EDIT: Here, let me show you. after 10 minutes putting my daughter on a makehuman. hideous. rofl everyone. Definately what you would call Proof of Concept. but that’s about it. codespeak for “a looooong way from production-ready.”.
you stretch the image to fit the mesh. sorta like sticking your face into a piece of saran wrap
Exactly, and it doesnt work, as in the monkey example. Features in model dont match features in photos.
And, excuse me, but Does your daughter know you are doing that with her picture?
Vertex Pusher:
No, but I did download it. Yeah its not the best, and considering you would think they would show the best it can do, it needs more work. That said, However, if you can do that in a matter of minutes, I think thats great.
Dont flame me, lol. The image itself that was used may be the fault, as the picture of the guy looks good to me.
And notice how the parts all match, thats good, yes?
Anyway, I still wonder about the coding of those functions. Any ideas? Will blender go that way too?
But like I said it doesn’t look like the actual geometry is being altered because they use an “image editor” on the pictures and then paste it to the model … Actual morphing of geometry based on photographs would have much better results then the sample they provided . I know there is research on creating 3D from 2D but As far as I know not even the “big boys” like Maya, XSI etc. have that functionality and neither does TrueSpace …
