How to render outline of shaded area or outline of shadow?
The reason I ask the question because I try to set a stylized result of shadow has outlines, but I can’t find any reference about how to do it.
Anyone know any solution or thoughts?
Thanks in advance. 
For a single light? Baking the edge of the light’s occlusion? With perfectly sharp shadows? (Because otherwise, the problem isn’t well defined.)
You could bake lighting and run an edge detect convolution on the baked texture. Wouldn’t cross seams well. You can do convolution in shader or compositing nodes if you want, but it’s not as easy as doing it in Photoshop.
For a smooth shaded mesh, you could get an approximation by looking at the dot product of the normal vector and the light vector. When the dot is ±1, they’re parallel (or anitparallel); when they’re 0,the normal is perpendicular to the light vector, indicating the “silhouette” from the light’s perspective-- the line that transitions from light to dark. What’s approximation mean? Well, smooth shaded normals means that the normals aren’t real, they’re fake, and so it’s not the literal silhouette, but as the mesh increases in vertex density, the smooth shaded normals approach the true normals, and eventually the error becomes too small to see.
How would I get the vector to the light? With 3 value nodes, with drivers to acquire the world-space position of the light. Then you combineXYZ, vector math/subtract from geometry.position of the sample, and run through a vector math/normalize.
Doing it that way is nice because it’s dynamic, if you want to move the light-- no baking.
An alternative here would be to simply adopt the view of the light and texture paint it, or project from view to a new UV map and create a texture for that new UV map, which would let you be more exact. That would work regardless of seams, regardless of smooth/flat shading. But not dynamic.
Hi bandages,
Thanks so much for replying! and provide such detail technical information. Very Appreciate!! 
To be honest, I don’t know what setup I should do. Do you have any resource/tutorial that I should watch to know about the vector stuff?
I try to get what Sable have done. (see the image below) You can see all the shadows has outlines. I think they have some code support to achieve that effect in engine. However I have no program background knowledge. If I could achieve that dynamic effect in Blender would be great! (if it’s “easy” to do). Baking is not my first option.
[the shadow outlines in Sable]
Thanks soo much!
If you want a render like that, presumably with an animated light, your best bet would be to use Eevee, and use shader-to-RGB (on a white diffuse) to find the border. It’s basically cell-shading with a black bar in the middle of your color ramp.
1 Like
thanks Bandages! This answered the same question I was going to ask!