Hi, I’ve been experimenting with custom vertex normals lately. Unfortunately it seems like cycles produces unsatisfactory results when rendering these, especially the more glossy the material is.
Have a look at the images below(go to the full res, it’s 3k in width)
The issue should be obvious in the fourth image, the cycles render. There’s a sudden, dark gradient at the edges. I’ve also added the three passes where the issue appears: glossy direct, glossy indirect and ao.
This seems to be related to cylces doing raytracing, as none of these issues exist in the viewport, the blender render, unity or unreal.
It’s happening with light sources, as well as with (only) environment lighing. I tried tweaking various light and render settings and different methods to do the custom normals. All without success. They look great everywhere except in cycles with glossy materials.
I wonder if other raytracers have the same problem? I’ve tested in substance painter’s iray (only environment lighting) where it is not happening(or maybe very very slightly, barely noticeable).
I currently don’t have access to any other renderers. If you have access to vray, octane, keyshot or whatever else people use these days and want to test, I’ve attached my test cube as well(cycles_and_FWVNs.fbx.zip).
Any input on this is appreciated. I have no problem recompiling blender with some tweak or some patch, if that is what it takes, I just need the tweaks and patches as that is beyond what I can do
Unfortunately, the suggestions - subdivide or edge split modifier - are out of the question, as they don’t work or break custom normals.
Attached is my test cube with overall smooth shading(no hard edges, aka a single smoothing group).
There’s a dirty fix to get around this by mixing the glossy shader with a matte one using the fresnel node as a factor.
This doesn’t work with the custom normals however, probably because the fresnel node is basing its value on actual face angles and not on smoothing, ha.
Did you try switching The path-tracing algorithm to “Multiscatter GGX” instead of “GGX” ?
This seems to fix it for me.
Multiscatter GGX is pretty new and was made to target such problems. especially for transparent materials, but also in your case.
I LOVE YOUR STUFF MOTH3R ! <3
Edit: Oh, nvm. Multiscatter doesnt fix the edges problem, sry.
Thought it did, but it really just solves the outer rim problem.
No idea on the edges problem, damn.
When faced with this problem for regular lowpoly stuff I tend to inset the faces that are supposed to be flat (no bending on a sharp glossy). I.e. on a beveled table top I want a complete smooth shading so I can’t use auto smooth (would make the last transition sharp) and if I do nothing the top face would also be curved (evidently when used on sharp reflections). If I inset the topface so it’s just a little smaller than the bevel start, I push the boundary outward and only the small region might show the error (typically it would be completely gone for all intents and purposes, even sharp reflections). So the inset face and the outer region (prior to the inset) have the exact same normal, so no actual smoothing gets done even if its enabled.
That works for me, but you may not always want to add “unnecessary” geometry. I’ve never played with custom normals, but I assumed it was there to help out for this problem.