Help with eyebrow-through-hair effect using Stencil/Depth in Blender NPR 5.2

[Kurtz02 (kurt0z2)]

Hi, I’m trying to recreate a common cartoon/anime rendering effect: eyebrows should remain visible through the front hair/fringe, while still being correctly occluded by the face/head and other geometry.

I’ve been searching and testing solutions for over two days almost nonstop without success. I’ve tried different versions of Blender, Goo Engine, Blender NPR 5.2, tutorials from 2AM and other videos/setups involving composition, stencil, ZTest, Depth Write, render order and separate passes, but none have produced the correct result.

Any Blender version or even Goo Engine is fine, as long as the effect works reliably.

I can either make the eyebrows appear through the hair or preserve proper depth occlusion, but I haven’t been able to achieve both at the same time, eyebrow in front of hair fringe, but behind head/body. I almost managed to achieve this effect, but the hair kept rendering the eyebrow from the other side ignoring the head.

Does anyone know a working method for this? Stencil, Filter Graph, custom shader, compositor, or any other approach is welcome, nothing seems to work at all.

Any help or working example would be really, really greatly appreciated.

[etn249]

Welcome!

This can be done with view layers and compositing, it will require rendering 3 passes. Sadly, the effect won’t be visible in the viewport though.

To do this properly, start by creating 3 collections: one for the body, one for the eyes, one for the hair.

Then, create 3 view layers (including the default, so 3 in total).

In the first layer, disable the hair and eyes collection, render the whole scene as usual but without those objects.

In the second layer, set the body collection to holdout in its collection settings, activate the hair collection but keep the eyes off. The hair will render alone on a transparent background and the holdout will ensure the parts of it hidden by the body are cut out of the render.

In the third layer, do the same but for the eyes (body collection as holdout, hair disabled, eyes visible).

The eyes and hair layers should be rendered on a transparent background for best results, or else the sky background would interfere with the compositing. To do this, I create a “transparent background” group: it contains a sphere with its shadows disabled that surrounds the scene and hides the world background from being seen. That collection is set to holdout and is used in the hair and eyes view layers to make sure those elements are rendered over a fully transparent background.

Now, go to the compositor and add all 3 view layers. Combine them with “alpha over” nodes. Choose the order of the layers so the eyes are over the hair.

–

A disadvantage of this method is that you will need some elements to be turned on or off in the viewport in specific ways and that can be inconvenient for working on the scene. I have a solution for this: create a 4th view layer and uncheck its “use for rendering” setting. That way, you can use that layer for your own viewport use and you can change the visibility settings without undoing the rendering setup.

Here is a scene where I do all that, including the 4th view layer for viewport use.
hair_composite.blend (157.8 KB)

[thorn (BA Staff)]

@Kurtz02

Does anyone know a working method for this?

Are you familiar with how to use the screen space node in goo engine?

ETA: actually, strike that. I think you will find it easier if you render the eyebrow material as an AOV and then mix it (transparently) over the hair in post.

[joseph]

In Goo, it’s very simple- no view layers or comp needed:

[Kurtz02 (kurt0z2)]

Thank you! This is actually very close to what I was looking for, and I really appreciate you including the .blend file as well!

Would it also be possible to control the opacity specifically where the two parts overlap, while keeping both objects fully opaque everywhere else? Basically, the overlapping area to have adjustable opacity while both parts still coexist there, rather than making the entire object transparent? Thanks again!

[Kurtz02 (kurt0z2)]

I’m not familiar with the Screen Space node or with using AOVs this way in Goo Engine, unfortunately. I actually don’t know how to render the eyebrow material as an AOV and mix it over the hair either. It does sound like it might be what I’m looking for, though.

[Kurtz02 (kurt0z2)]

Unfortunately, I tried this setup and got the same result as in my screenshots. The eyebrows still renders over the body and the hair renders the eyebrows through it in the side view, ignoring the body occlusion through it in the side view, ignoring the body occlusion. Thanks for trying to help!

[etn249]

Yes, this can be done in the compositor. You would have to use the alpha layer of the hair as a mask to decide where the eyes are visible and where they aren’t. Then, you can blend between the version where the eyes are hidden and the one where they aren’t.

Here is the new compositing setup. It’s a bit more complicated, but it doesn’t add anything to the scene or to the render time. The mix node at the end is the intensity slider, it mixes between the version where the eyes are hidden and the one where they aren’t.

[Kurtz02 (kurt0z2)]

That worked perfectly! Exactly what I was aiming for. Thank you so much for taking the time to update the setup and explain it, I really appreciate the help!