Smooth Normals On Double Sided Alpha Mapped Textures

Now this has been a chore.

For the last couple of days, I’ve been studying up on how to make trees, plants, grass, and whatever other type of foliage look as smooth and detailed as doing everything with pure geometry. You think it’d be easy, especially considering how often people do it with seemingly no problems whatsoever.

Unfortunately for me, I’m not one of those people. Every attempt I’ve made has been filled with failed experiments, and rough compromises to get the look I want. For reference, I’ve been using what I’ve learned from these two pages for my trees and shrubberies.

Airborn Trees
Blender Specific Tutorial For Spherical Normal Shaded Trees.

With that latter one, you’d think I’d knock it out of the park. Nope!

Here are my problems. On a two sided, non backface culled object, I can get the top halves of my trees looking decent. Unfortunately for me, the backfaces are almost always extra dark, no smooth gradient, and it stays that way no matter what I do. The Normal Edit modifier doesn’t seem to do much of anything with the sphere trick, and editing the coordinates manually can get me smooth shading, but I sacrifice accurate lighting.

I’ve tried turning on backface culling, duplicating the object, and flipping the normals. That nets me about the same results, and usually flips the faces back to their “proper” orientation when I run it through the modifier.

Now I’ve come close to getting what I want. When I turn on backface culling, and carefully position the bushes I’ve made, I can make it look good, so long as I look at it from one specific angle. The dark shades are properly dark, and the lights are all scene accurate. If I start rotating around it though, whole chunks of my object will disappear as I come across the rear faces. It’s such a tease, cuz I’m so close, but I need a technique that works reliably without spending 45 minutes just to get something that looks good only in specific circumstances.

…and of course this has run up against my black contour trick. I really do wish you could turn backface culling on per object, and/or per material. I found a technique for the latter, but it doesn’t seem to work in Eevee. Eh. I’m still learning. :stuck_out_tongue:

Here’s a shot I fixed up showing how things should look when they’re firing on all expected cylinders. The bushes look great right now, but if I turn off backface culling, they’re filled with tons of ugly blotches. The grass is all geometry, all three million polygons worth, cuz I can’t manage to make x-planed grass without it look like pureed butt.

Help me achieve my dreams of pretty foliage!

To show you all what I mean…

Backface culled

No backface culling

I’ve got these aligned for optimal viewing, and they’re dense enough so the effect isn’t quite so noticeable, but you can still quite easily see the shading errors and black leaves on the 2nd shot. For any object that isn’t appropriately dense enough that I can get away with one sided polys, it looks terrible.

Awesome image you are creating there.

The problem is a little bit hard to diagnose without taking a closer look at the geometry and material definition. Can you take a screenshot of the material definition? Can you take one (or multiple) of the geometry of a problematic leaf? I’m looking for anything unusual with the latter (flipped normals, self-intersections, inner faces, duplicate faces).

I’ll do you one better, and send you the models and texture. It’s a 2.8 file, so you’ll need to either open it there, or append it into 2.79. (2.3 MB)

Thanks, and sorry for the delay. Somehow I didn’t find the thread anymore. But now I checked your blend.

What causes the odd behavior is the split normals. I can’t determine from the blend how exactly you created these, but I guess it was some variant of what they did in the tutorial. The situation was, that quite a few leaf planes had the split normals almost inverted, which results in the side opposite of the light source being lit, while the one turned towards it being shadowed. I think the trick with the spherical normals works in the tutorial, because the leaves there were oriented orderly around the tree and thus would not be inverted, while yours are random. Don’t how easy it is to make it look good, but a start would be to orient all planes towards the ‘outside’ of the bush, before doing the normal edit.

1 Like

Somewhat. I started with a edited UV sphere, made a few leaf planes and bases, distributed them as particles on the object, made them real, then deleted the underlying sphere. It seems my quick shortcut is what’s causing me the most problems.

I’ll try and place all my leaves by hand, see how that turns out.

Thanks for the help. If I have any more questions about this (and I probably will), I’ll hit you up with an @omgold

So after all these months, I’m still pretty much exactly where I’ve been. Getting smooth alpha mapped trees has been the one thing I’ve struggled with the most, and it’s something I just can’t seem to do.

I sat down yesterday to really hammer at it, and figure out exactly what I’m not doing that everyone else is. To start with, I jumped into the Book of the Dead asset pack that the Unity bunch kindly provided for free. I found a way to export the models there as .fbx files, loaded them into Blender, and spent some time studying them.

…and they’re not doing much more than what I’ve done previous. I picked up some nice techniques, but it doesn’t seem they’re doing anything massively different. The thing is, despite having access to all the same models and textures, I cannot get these trees to look as nice and smooth in Blender as they do in Unity. I can’t even get them to look as smooth as the trees you’d see on a random 5 minute tutorial on Youtube.

Since a picture is worth a thousand words, here’s a couple for you to show what I mean.

This is a tree in Unity.

This is the same tree in Blender.

Nothing I do in Blender can make it look even half as good as that first shot. I’ve gone through tutorial after tutorial, even gone through the ones I linked to above again and again, and I cannot seem to reproduce their end results. I’ve played with the normal maps, inverted them, reverted them, flipped normals like they’re going out of style, backface culled, doubled up on geometry, endlessly played around with the normal edit modifer, and even tried weighing them. It always turns out to be some variation of the above.

And they make it look so easy in all the Youtube videos I’ve watched. Just 2 clicks, and BAM. Nice trees. NOT FOR ME!

So I figured two things. Either editing normals as such is broken in 2.8 (not that I had much success in 2.7 either), or I’m missing something obvious.

I just want to make some trees! That’s all I want! Is that so wrong?

On a guess, could it be the way I’m setting up my transparencies in the shader? There has to be some weird reason why I can only achieve hard edge chunkiness here.

Gonna bump this one more time, just in case it’s missed.

If no one knows the answer here, I’ll bug out to Polycount or something like, and if I find a solution to my problem elsewhere, I’ll update this thread with what I’ve found.

  1. The normalmap should be Non-Color Data.
  2. Play with Properties Editor > Material > Settings > Alpha Clip Threshold or use Alpha Blend/Alpha Hashed
  3. Don’t just use a Sun Light, also use an HDR Image for lighting.
  4. Leave Specular at 0.5.
  5. Use a Roughness Texture with varying grayscale values.
  6. For best rusults google for Cycles tutorials on vegetation shading.
  7. Always upload a blend file, if you want the best solutions.
1 Like
  1. Did that. Made things marginally better, maybe?

  2. Did that. It does look slightly better with Alpha Hatched and Alpha Blend, but still far from my intended goal.

  3. I’m using one of 2.8’s built in HDRs. It looks even worse with direct lighting on it.

  4. Set that, and it just made the brighter edged branches look like they were smeared with vaseline.

5 & 6. I’ll give it a try.

  1. Normally, I’d do that. Though in this case, I’m not exactly sure I can, considering this isn’t one of my models. It’s a model I’ve ripped from elsewhere (The Book of the Dead Unity assets specifically) so I can experiment with alpha masked trees using an object I know is already well built.

…I gotta say, the whole experience has been educational. I’ve learned quite a bit about what doesn’t work. Problem is, I haven’t figured out what DOES yet.

For poops and giggles, I might try to import another tree to see if maybe, MAYBE, there’s an issue with the assets used in particular model that make it chunky in Blender.

The steps are not optional, all need to be done to get decent results and only after you did all will the model look right. It doesn’t work like I try just this one step… crap, it didn’t work, ok I’ll never do this again. Next step…
I don’t have Unity installed so unless you send me sg you’re on your own.

I could send you the model via PM. You cool with that? That way I can say I’m not putting any one else’s IP up for all the public to grab.

This is purly for learning, no bad thing will happen to you. It’s a free download, any body can grap it, if they want to.

Just for the record, I didn’t feel like downloading Unity, then downloading another big-ass package, would take too much time, plus it was a while since I last used Unity for a short time.

Yeah, I know. Better safe than sorry, though. :stuck_out_tongue:

Gimme a sec to zip everything up.

Don’t zip up everything, I just need one model.

Just the base blend file? No textures?

Don’t leave out the textures, their the most important. I meant I didn’t want the the whole 2GB package.

Ha! No, I’m not going that far. Though the zip is a little larger than what BA allows through an upload, so I’m setting up a Onedrive Link.

If it’s under 30MB you can upload it to