PBR Shaders and Nodes that use Standard PBR Textures/maps

[AustinC]

EDIT: THIS ENTIRE THREAD IS IRRELEVANT WITH THE NEW PRINCIPLED SHADER IN BLENDER 2.79!

I have been working very hard to create quality PBR shaders that:

• Work well with PBR maps from other software like Substance Painter and Toolbag

• Look accurate
  [LIST]

• Fresnel curve shape should change with IOR - this causes interesting edge variations

• Glossy surfaces should still conserve energy when not rendering caustics

• Conductors should have complex IOR (N, K) (input colors could be used to calculate N and K for artistic control)

• Shaders should react to roughness properly (NOT what CynicatPro does)

• Use real world inputs - the artists can map input values outside of the shader node if needed

• Dielectrics should be able to tint their reflections (not quite the same as conductors)

• Glass should disperse light, but doing so should follow energy conservation and respect the spectral composition of the incoming light

• Shaders should be combined in a physically based order and manner

• Be efficient

• Use clever modular shaders to create materials only as complex as is needed (not ubershaders)

• Use as few shader nodes and mix nodes as possible

• Be easy to use

• Fake refractive caustics to a reasonable extent (to avoid needed real caustics)

• Fake reflective caustics at least enough to conserve energy

[/LIST]

I made my basic shader components for dielectrics, conductors, cloth, glass, etc. But, I needed to make a custom node group to handle typical PBR maps. Since conductors require two color (or vector) inputs, the conductor shader would not be practical to use in such a manner. So, I only used a dielectric shader. My dielectric shader supports tinted reflections better than most PBR metal shaders, so it may be sufficient. I just needed to map the inputs correctly. The only thing that could be an issue is the glossy algorithm in the dielectric. The dielectric uses GGX while a metallic surface would use Beckmann. If that is a problem, I just need to add two more nodes.

I downloaded the assets here to test my PBR shader for the metalness workflow.
http://artisaverb.info/PBT.html

I imported the model into Blender, linked in my PBR metalness shader, and setup the model’s material as follows:

I set all the textures to “non-color data” except for the albedo map, which I left at “color”. The node group contains 3 shader nodes, 2 mix shaders, and 1 add shader.

This is the result with no material tweaking. It took 12:57.82 to render compared to 10:45,54 for a plain glossy shader.

This is the original rendered in (I believe) Toolbag:

Attachments

[Kremor]

Looks photorealistic. Keep uo the good work.

[Spirou4D]

Hi @AustinC,
Always a very nice work ! Congratulations.
And now you share your setup, Wooow!
Thanks you so much. Have a nice week.
Friendly.
Spirou4D

[lsscpp]

@AustinC"

I have been working very hard to create quality PBR shaders that:

• Work well with PBR maps from other software like Substance Painter and Toolbag
• Look accurate
  [LIST]
• Fresnel curve shape should change with IOR - this causes interesting edge variations
• Glossy surfaces should still conserve energy when not rendering caustics
• Conductors should have complex IOR (N, K) (input colors could be used to calculate N and K for artistic control)
• Shaders should react to roughness properly (NOT what CynicatPro does)
• Use real world inputs - the artists can map input values outside of the shader node if needed
• Dielectrics should be able to tint their reflections (not quite the same as conductors)
• Glass should disperse light, but doing so should follow energy conservation and respect the spectral composition of the incoming light
• Shaders should be combined in a physically based order and manner

[/LIST]

looking forward to see what you come up with these…

[AustinC]

Thank you. Unfortunately, there is a reason why this is in the Blender Tests forum. I am still having an issue with the Fresnel; there is a mathematical instability in the Fresnel equation for dielectrics. Dielectrics can’t be more reflective than about 36% (EDIT: roughly speaking; a darker ring forms around the edge). So, the “metalness” must blend between a dielectric and an actual conductor, not just force the dielectric reflectivity up. I think that I have a solution, though.

EDIT: Yes, problem solved.

[lsscpp]

@AustinC

… Dielectrics can’t be more reflective than about 36%…

Can you please point us to the source of all this goodness (technical information i mean…)

[AustinC]

@lsscpp

Can you please point us to the source of all this goodness (technical information i mean…)

It’s nothing based upon experimentation, at least not that I know of. It has to do with the Fresnel equation. For IOR greater than ~4.0, the curve will begin to darken for higher angles. Some materials have this behavior (only conductors, that I know of), but not as severely. With increasing IOR, you will not even notice the curve’s sharp rise to 1.0 at 90°; the edge will just darken.

The dipping of the curve can only be controlled by using a complex IOR. That is where N and K come in. N is the real component of the IOR and K is the imaginary component. Only conductors have complex IORs, and that has to do with physics. Moreover, an N of less than 2 must be used; it’s just how the math works.

I read a paper that maps N and K to two new variables, r and g. r is the reflectivity at 0°, and g directly controls the dip in the Fresnel curve. From there, I made an equation to find the g required for K=0, which is purely dielectric behavior. I use this equation for g until r=0.2. For r=0.2…1.0, I use an equation for g that maintains a smooth Fresnel curve until r=1; which necessarily results in K>0.

EDIT: To answer your question, I made that statement based upon my analysis of the Fresnel models. There is no specific source. Also, to help in my analysis, I graphed the Fresnel equations in Excel, so I observed this behavior. I also observed this behavior in Blender by putting a Fresnel node into an emission shader. If you want some of the sources for what I say in this post, those I can give you.

[AustinC]

That wasn’t my full node setup; it was just to show how simple the node group was to use. It was just one node with some image maps connect, nothing more. The full node setup would take pages. So, you didn’t miss anything.

I am still exploring the actual roughness implementation. There is not much literature on this, especially regarding Cycles’ implementation. The largest problem is that Cycles cannot properly implement roughness or Fresnel. It is just a matter of which hack is better. So, I may still fall back to CynicatPro’s hack, although I’d feel better knowing where he got it. Here is a thread talking about it: https://www.blender.org/forum/viewtopic.php?t=27455

Currently, I have created the full reflection portion of the Fresnel node from scratch with support for complex IOR inputs (N and K). It behaves just like the internal Fresnel node, unlike the common approximations. However, because it doesn’t have the refractive portion that the built-in Fresnel node has, it will work for single-sided geometry. (EDIT: I forgot to answer you straight. That is basically the effect that I will get, but I will get it by generating those curves mathematically using the equation and measured properties. I have basically made this from scratch, but without the anisotropy: https://cgcookiemarkets.com/all-products/metal-bsdf-for-cycles/)

You are absolutely correct. I made a single node group for metalness workflow, and I had to take some shortcuts to get it to work as intended. One of those compromises was not using the N, K metal shader. That is a fancy shader that would be used specifically for a certain material. And, I may need to take other shortcuts as well. My greatest challenge has been linear behavior with respect to the inputs… I’m trying really hard to keep the number of shader nodes to a minimum.

The IOR variations are actually very easy. I made a Fresnel node that takes reflectivity as an input, so I just use one for each color channel of the reflection color. Then, I combine the color channels. The problem is mixing it with diffuse.

No problem; I am very critical of my own work anyway. That results in good work. I have had many difficulties. My main approach is to make the best shaders possible, within reason. Then, I will begin simplifying and approximating where possible. But, having a high-quality reference will let me know what I am compromising. And, some things might just be a losing battle, so I will need to see how it goes. Currently I have a lot of good work, but I don’t know how useful it will actually be.

[AustinC]

I actually haven’t seen that thread, although the fundamental assumption that it makes in the first post is incorrect. Diffuse behavior is fundamentally different from glossy behavior. However, it is correct in saying that all the renderers are wrong. Roughness models are incorrect for both of the shaders.
In the future work section at the end of this paper from Disney, they mention one of the issues with current roughness models, specifically energy conservation:
http://blog.selfshadow.com/publications/s2015-shading-course/burley/s2015_pbs_disney_bsdf_notes.pdf
There is nothing that we can do about that in Cycles, especially since Cycles lacks a lot of features under the hood. However, mixing diffuse shaders with glossy shaders helps a little, IF the Fresnel is good enough. The main issue is that Fresnel should decrease near the edge for a rough material. Here is some measured Fresnel data:

That dotted line is the Schlick approximation for the Fresnel reflectance equation. Notice how the edge reflectance decreases at grazing angles. However, it decreases in such a way that there is a “bump” near the edge. This maintains the Fresnel effect, but it removes the harsh value of 1 at the edge. This is much preferred over simply flattening the Fresnel curve. Unfortunately, this is a function of the “difference” angle, which we can’t use in Cycles, so I can’t be sure of the physical implications of this. But, regardless, I am finishing a model for this behavior (I am just finishing the model development in Excel). I will see how it looks and judge. In the very least I can round off the sharp spike to 1 at the edge that is quite idealized and unrealistic.

[AustinC]

Actually, it’s unfair for me to say that Cycles “lacks a lot of features”. It is just how Cycles is made. It must decide how a material reacts to light before it knows where the light is coming form. I’m not sure why it behaves this way (DIT: It is supposed to be fast, but other engines do it correctly faster.). In the end, the implementation that CynicatPro uses may be best, but it needs a better response curve than just linear. Again, I really wish that I knew where the method came from.

EDIT:
Well, I’m not wondering that much. I think that he got it from this paper:

Sairiala_Joonas.pdf

That paper was published in February 2015, and CynicatPro uploaded his Fresnel tutorial in August 2015. CynicatPro makes many references alluding to the fact that he didn’t come up with anything himself. And, it is painfully clear that he doesn’t comprehend most of what he learned/does. Unfortunately, Joonas Sairiala does not give any scientific justification for this method.
However, this image of a better Fresnel effect (using the difference angle) looks very similar to blending the surface normal with the incident vector. You can tell from the image that the Fresnel effect should be flatter at the edge for higher roughness than a simple incoming vector blend will achieve. I don’t know if that flattening effect is worth modeling. Regardless, if the blending method is used, the blending curve needs to be better. I may use the image below to calibrate a blending curve. It will be good to have several models to compare.

EDIT 2: Blending with the facing vector just doesn’t look good. For various reasons, I’m attempting to adjust the Fresnel curve rather than to recreate it with a curve fit. My equations are getting close to the picture above, but they still have edge artifacts that are not trivial to remove.

[lsscpp]

@AustinC

I actually haven’t seen that thread, although the fundamental assumption that it makes in the first post is incorrect. Diffuse behavior is fundamentally different from glossy behavior…

Of course they behave differently, but actually that nodetree tries to deal with a more general concept of surface, with a slider that just tells how rough it is at a microscopic level. The behaviour should be intended as a consequence. And of course, it’s fine with the easy tools i found: two shaders and some eyeballed way to mix them.
Probably the best thing we could ask for are two new shaders: dielectric, and conductive, with proper fresnel and ior, as found in many other renderers

[AustinC]

@lsscpp

Of course they behave differently, but actually that nodetree tries to deal with a more general concept of surface, with a slider that just tells how rough it is at a microscopic level. The behaviour should be intended as a consequence. And of course, it’s fine with the easy tools i found: two shaders and some eyeballed way to mix them.
Probably the best thing we could ask for are two new shaders: dielectric, and conductive, with proper fresnel and ior, as found in many other renderers

True, and we have to do what we can with what we have. I just meant that it can’t work for something like PBR because glossy behavior is too different from diffuse. The first image that I rendered with the shotgun has bright edges at grazing angles that look like dust, but it is actually a “glitch” where the Fresnel curve made the edges diffuse rather than glossy. The albedo was the same. So, even though blending a rough glossy with diffuse looks decent in some cases, its behavior isn’t predictable for all lighting conditions. Consistent behavior is one of the main advantages of PBR.

Unfortunately, we can never have a dielectric and conductive shader like you are thinking of, not without greatly extending Cycles. Many people could write shaders with OSL if it were possible. The problem is that the equations require information that Cycles does not calculate. The Fresnel equations require the vector between the incoming light and the outgoing light. This vector is also affected by the roughness. Because Cycles only traces the light FROM the camera, the rays hit a material before the render engine knows where the light is coming from. Using the tangent is a cheap hack. With Fresnel, we are left with faking it as well as we can. Even if it is incorporated into a shader with some kind of roughness implementation, the results still will not be great, although I don’t really know what those results would actually look like.

[AustinC]

Improvements:

• Fixed the Fresnel curve shape issue for higher IOR. This gives proper grazing angles, which has changed the material enough that this lighting isn’t good.
• Made the metalness shader change linearly with the metalness parameter.
• Implemented an artistic roughness effect for the Fresnel. I used the Fresnel test scene and created the comparison render using my rough Fresnel node. You can see that the edges still maintain extra gloss. I desired this behavior. It’s going to be unrealistic either way, so I made it look good, or at least what I think looks good.

Here is an updated render.

Again, the lighting needs work, but it seems to correctly implement the PBR maps.

Attachments

[AustinC]

So, it’s been a while. I’ve made much progress. First, most Fresnel equations in the art world are rough approximations. This is a graph of several Fresnel models at IOR=1.5.

The red line is the common approximation. The blue line is the exact Fresnel equation, which Blender’s Fresnel node uses. The approximation is always 1.0 at 90°; real materials don’t have a Fresnel of 1.0 at grazing angles. If you scale down the peak at 90°, the slope is too steep. So, I fit several Gaussian functions to the exact Fresnel curve.

The purple line is a fit based upon the incident reflectivity and the grazing reflectivity. It is very robust and works well.

The green curve is a better fit. But, it only works from IOR=1.2 to IOR=3.2. It is more intense to calculate than the purple approximation, but it gives better results.

[AustinC]

I have updated renders. This first image is the metalness PBR shader as it was with a little compositing added. The roughness response wasn’t good, so the material was too smooth. Also, the grazing angles were too glossy. The dust is hard too see, and everything is too dark. This took 1h02m to render.

This is the updated shaders. I made the metalness perfectly linear. I optimized a few things. And, I implemented the new Fresnel nodes. This shader uses the purple approximation from the graph above. The rendering settings were exactly the same as the rendering above, but it looks quite different. You can see the dust because the glossy doesn’t overpower everything. This took 45m to render, so the optimizations really helped.

But, that image doesn’t have the roughness fix implemented. I redid the roughness correction factor to be more modular, and it works well. This is the final test render of this model/scene.

At this point my basic PBR shaders are done. I should have all the tools and building blocks to quickly texture most things. Unfortunately, making test renders is so difficult because my desktop PC died nearly one year ago. So sad, but I have accomplished quite a bit on my $380 budget AMD laptop.

[AustinC]

I am concerned that the Fresnel’s roughness will be too sensitive. I had to make it very sensitive to pass that roughness test with the red spheres. If that is the case, I will use a less sensitive Fresnel roughness. Because Blender doesn’t actually have the half vector, there is only so much that can be done. In fact, I may simplify the Fresnel roughness anyway and make it behave linearly. These are minor tweaks, though.

EDIT: After playing with it, I definitely need to make some changes. The roughness is far too sensitive. The roughness needs to affect the incident reflectivity. So, I am going to use just the purple model, and I will add roughness to the equation.

[AustinC]

I tweaked the roughness factor to be smoother and look better. It doesn’t make up for the Fresnel using the wrong vector, but it works alright. I made the roughness decrease the incident reflectivity to a minimum of 0 and decrease the grazing reflectivity to the smooth incident reflectivity, which is 3% for the metalness node. Increasing the metalness a small amount lets you increase the reflectivity more without affecting the material behavior too much.

I’m not too picky about realism, but it needs to look good. I’ve hacked the roughness to look reasonable, but using the wrong vector for Fresnel produces odd shading that doesn’t look good. This odd shading is very noticeable in animations or uneven lighting (all professional lighting). But, I may need to just accept this.

This tries to fix the problem.
This is not my video or shader!

You can see how much of a difference it makes. The render looks so much better when using the correct Fresnel. Fresnel should only apply where light hits the object, not just around the edges. But, the method in this video has other problems that may look even worse.

The only real solution would be to use a render engine with bidirectional path tracing. I may begin exploring other render engines. Until Cycles has at least an option to use bidirectional path tracing, it’s hard to take it seriously. Metropolis light transport would make Cycles a very strong competitor to the best of them. I really like how Cycles works, and its integration in Blender is great. It is fine for metal, but I’m trying to do assets and animations of things that aren’t metallic. The only other option is to disable Fresnel. Maybe I will make some renders with Fresnel disabled.

[lsscpp]

The video shows a hack that totaly breaks reflections.
Fresnel equation in Blender is just ok for PBR use: the cause of those black artifacts, is that suzanne has smooth shading but too low faces count (subd = 1, rise it to 3 and they disappear).

About yor experimentations, i’d like to know how you obtained those fresnel approximations (green and purple).
And by the way, since it looks like you’re taking a technical approach, showing the renders with glares, besides making the image cooler, makes it difficult to evaluates how your shaders work on a technical side. On the other hand, if you are seeking aestethics goals “why caring about all that math”?

[AustinC]

@lsscpp

The video shows a hack that totaly breaks reflections.
Fresnel equation in Blender is just ok for PBR use: the cause of those black artifacts, is that suzanne has smooth shading but too low faces count (subd = 1, rise it to 3 and they disappear).

About yor experimentations, i’d like to know how you obtained those fresnel approximations (green and purple).
And by the way, since it looks like you’re taking a technical approach, showing the renders with glares, besides making the image cooler, makes it difficult to evaluates how your shaders work on a technical side. On the other hand, if you are seeking aestethics goals “why caring about all that math”?

Thank you. That helps explain it. I have since studied how 3D smooth shading is calculated. I had some odd artifacts from it in a different thread.

I will gladly post everything as soon as I build a new PC. It is very complicated, so I want to make a PBR tutorial that covers it in detail. Basically, I fit two Gaussian equations to the actual equation in two different ways. It was extremely subjective. The fits have several variables that affect their shapes. I wrote equations that drive each of those variables. The driving equations are functions of the roughness and IOR/reflectivity.

That is a very good comment regarding math vs appearance. My approach began with math to achieve realistic behavior more quickly and intentionally. The mathematical model itself is just an approximation, an incomplete one at that. I just had to mimic the behavior with a fast equation that looks good. In the end, only the looks matter.

As for your question, I added the glare just because I wanted to show off beautiful renders. My technical test renders actually used that sphere scene, but it was very difficult to use. So, I made my final tweaks without them. When I make the tutorials, I will show everything with test renders and an occasional scene. This entire thread has been very rushed because I have been so busy and because my laptop takes so long to do anything.