Hello everybody, this is my first post on any sort of 3d forum ever asking a question. This is one situation that I couldn’t solve by extensive googling.
PROBLEM: Animating a dragon with thin wings. Said wings’s membranes invariably self-intersect while animating, producing black faces when rendering with Cycles and normal map applied. (See images)
Note that using any Cycles or BR shader WITHOUT normal map, intersecting faces don’t render as black. But I absolutely need the normal map, or a bump map. So far I’ve been using the “Simple PBR shader” by Jason Clarke, using Substance Painter textures.
What I’ve tried:1) Using a Geometry node to render back faces as transparent. Simply put, this doesn’t work.
2) Manually merging wing membrane vertices in between “fingers”. This helps a little but doesn’t completely get rid of black faces, and the result still looks like crap.
3) Using various other shaders. Basically, any Cycles shader with my normal map plugged in results in black faces.
What I can’t do:1) Can’t use another dragon model or significantly alter this one. This one’s unwrapped, textured and rigged with a complex rig and I must use it
2) Can’t use Blender Render because the renders look very poor compared to Cycles.
3) Can’t manually animate vertices for every frame so they don’t intersect (duh)
4) Can’t discard the normal map because it’s crucial for the nice look that I’m after. I imagine I could use a Bump map or something like that, but I doubt it will solve the problem and besides, why the hell can’t I use a Normal map?
Possible solutions that I thought of, but don’t know how to actually implement:1) Somehow make the mesh automatically not self-intersect during animations. Some kind of automatic relaxation based on intersecting geometry? Perhaps a minimal allowed distance between polygons? Googled the hell out of this as well, with little to no results.
2) Somehow make the damn Cycles render the intersectiong/backfacing polys normally, as BR does.
3) Anything to make the render look decent. Maybe rendering a shadeless basecolor render and shadows separately? At this point I’m starting to consider this option, the pain that it might prove to be.
Literally any thoughts on this matter would be grealy appreciated. Thank you! I will answer any questions and promise to contribute good answers in the help forum to the best of my ability.
EDIT: since yesterday I’ve managed to find a workaround. I used a color-mapped Displace modifier to artifically thicken the wings to avoid backfaces protruding through the front. (I have to thank Nathan at Sintel movie blog for this idea). This yielded a decent result, but my original question still stands, who knows what other artist might stumble on this problem and not be able to use the Displace thing.