I am making multiple photogrammetry scans (with reality capture) of single same person with different facial expressions (happy, angry, sad etc…), head with stay the same as it is same person.
I want to use shape keys even those are all different meshes.
What is the best way to merge those various scans to the single head and animate them using shape keys or maybe you can suggest something else?
As I understand they need to have same topology? What is the best way to make different scans have same topology? Any good workflow for that?
Hello and welcome to the community! you would be better off transferring the face shapes to the neutral face base scan, you can do with this free addon https://mmemoli.gumroad.com/l/tOKEh
only thing is… it need to have same topology. What is the best(fastest) way to make different mashes (in my case expressions) have same topology as neutral expression face?
you mean gradually morphing shape on the uvs or model? in the case of the model thats for what the shape keys are for, if you notice it has a slider for exactly that purpose for gradualy changing the shape based from the basis shape key to the custom shapekey
that same addon has a uv transfer option iirc, if that doesnt work you would need to do a raycast transfer in which is done using the baking panel in cycles to bake the textures to a different object and uvmap, as long as the object is closely matching the same shape of the target object…
Assuming you have 3 different face scans all with different topology, I would do this…
Make 1 neutral face mesh which would basically be a retopo of a neutral face scan. Then use a shrinkwrap modifier on it to target a new face scan, apply the modifier as a ‘shapekey’. Then rinse and repeat as needed…
Randy
Edit: as @joseph stated, sometimes there is no magic button or addon quick fix… sometimes you have to do the hard work manually…
Using an automated remesher wont exactly fix what you want but it will generate a cleaner mesh for transfering a shape to other remeshed object with should have similar topology. In other case it would be cleaner and accurate if you did a single remesh from the base face scan first (or much better a manual retopology for more control of the result) with this paid addon https://exoside.com/ and make duplicates of that resulting mesh and then “transfer” the shape with this other one https://blendermarket.com/products/softwrap
Or you can commission a freelancer to do it for you (i am available )
Yes, shapekey have slider and I want that function. Morphing from neutral to smile, shocked etc. As i understand shapekey only work for same mesh with same polycount and work based on vertex changes. I’m i correct? Good when doing mesh changes or sculpting, right? But if i have to different meshes (even though they are mostly similar) it wouldn’t work?
About uv transfer from that addon or raycast transfer, is there any video that i can see how it work? I will have one same head with neutral face and different expressions exported from reality capture.
I know I can do it by taking reference photo and sculp my main mesh according to them but I wanted quicker (more realistic) method.
This softwrap looks like something i need, but price is little to high. Anything similar slightly cheaper and newer?
My first thought was about using this method but would like to see workflow video.
Don’t know if i would need to attach those faces to “neutral head” before shrinkwrap or it can work with only part of head(open mesh) (face, or sometimes just eyes) because those part will only be changed.
Shapekeys are on the object and yes they are based on vertex changes. Shapekeys cannot change the vertex count, just like you can’t change the vertex count in object mode. You can only change vertex count in edit mode.
Probably not going to find workflow video of just using shrinkwrap, just ones of addons people have created.
Probably want to do the whole head because you’re going to have a whole head mesh, right?
Across the top there are 3 cubes, standard cube in center is Neutral. Cube of left is Nose - because it’s like a nose sticking out. Cube on right is Mouth, because it’s like a mouth open shape. The bottom object is a UV Sphere that I scaled up in size. I positioned the sphere on the Neutral cube and used shrinkwrap on it, applied as shapekey. Repeated the process for the other 2 cubes. Now I have 3 shapekeys on sphere. Turn on the neutral shapekey and the sphere forms the shape of the Neutral cube.
Sure it’s very rough example, like trying to fit a square peg in a round hole. Using similar topology would give much better results. Like instead of a UV Sphere, if I used a moderately subdivided cube, the results would be better.
the uv option should be same use, but in this case i think is gonna replace the original uv with the transferred uv…also you may not need to transfer uv from the others and their textures if the neutral character is already textured and unwraped right?
no that’s the only tool that works for that, in the other hand you would have to do the shrinkwrap, relax and other adjustments manually if know to manipulate the blender modelling and sculpting tools.
and i checked your file. Even though both in the video and in your file you dont even have similar topology neither vertex count but it works.
So maybe i’m just overcomplication something not that hard (in the few weeks i’m planning to do the photo scanning and I’m just preparing to see should i only scan blank face and shoot just reference photos for sculpting or to scan all expressions)
I usually texture paint my models… can i expect some problems with color/diffuse/normal map or it will just follow shapes?
And last thing… let say I done it, rigged my model, and make some running or jumping animation, cause i’m not doing only face but full body scan (but with hair, clothes done in blender) is it possible to export mid-air jump frame as single 3d object. I mean like regular 3d model for just this single pose frozen in time with clothes and hair in air?
If you understand what I need. I know I need to convert hair to mesh, merge clothes with body, remove rig, it’s completely other topic, but just want to know is it even posibble to be done in blender?
I think maybe you are here. I’ve not worked with scanned models before, I’ve opened them and seen they were incredibly dense, like 500,000 + verts. You don’t need that many verts in the mesh you are shrinkwrapping, you only need enough verts to show the details. I would think a face with 2000 verts should be detailed enough to shrinkwrap and capture all the detail of a 500k vert face scan.
So what are you really doing here?
Face scan for emotions is one thing, full body scan and rigged character is another thing. If that’s what your doing, then look to motion capture instead. Warning - it’s not as easy as they lead you to believe from what I have seen, but it’s workable.
Clothes in blender with the cloth sim, I haven’t had good results with it. I don’t think it can be converted to a solid mesh. I don’t know if it can be exported, guess it would depend upon the format you are exporting to… maybe…
I always simplify my scans and bake maps (normal for detail), so my models are usually up to 100k polygons, some examples here:
Those are only posed models, so I want to up my game a bit.
My plan is to combine rigged body with corrective shapekeys for body only and use shapekeys (without face rigs, only necessary head armature) for expressions on my own simplified scans. Probably create pose library.
)
