Cycles: Experiment with shadow catcher

Here’s a test build of Sergey’s work with shadow catcher arc patch D1788 =
arc_patch_D1788_Win7.x64-4dbc4af

Test blend

Updated 11.18.16
Mainly rebasing against latest master, don’t expect any difference in behavior
@Lukas Stockner, i’ve tried implementing your idea of using non-MIS-weighted
numbers for the shadow, but there were following issues:

  • It made BPT somewhat darker shadows than with PT
  • I’m not sure we can store MIS weight in BsdfEval anymore: this weight is different for different terms in the BSDF eval sum, so don’t think we can divide final sum bu overall MIS weight to get number we want?

@Brecht Van Lommel, can we start some discussion about what is the best approach to get
this into master? What i have in mind is:

  • Figure out whether i got Lukas’s idea right.
  • Make this an Experimental option, so we keep some margin about making possible changes in the render results if we’ll re-consider some parts of the change to work better with denoiser.

Updated 10.1.16
Fix typo in light sampling
Was causing missing shadows in files with no world in certain cases.

Updated 9.27.16
Rebase against latest master
More real work is upcoming.

Updated 08.31.16
Fixes for branched path traders

Was missing some scaling factors here and there. Also did some simplification
of what needs to be stored to have properly calculated shadow.

Once thing i still didn’t manage to solve is difference between PT and BPT
when catching shadow in a scene lit by only world AO.

Fixes for AO

BPT gets closer to PT with change and it’s seems logical thing to do anyway,
but somehow there is still some difference in AO between BPT and PT.

Also disabled shadow tricks for split kernel OpenCL – it’s a bit tricky to
implement them in this kernel.

Updated 08.26.16

  • Solved issues with just a background light
  • Solves issue with colored lamps
  • Solves issues with GPU
  • Both PT and BPT are expected to work

Updated 08.26.16
Update self-shadoing part of the patch.

Details from Sergey

DISCLAIMER: This is more a code dump of a local branch, not somewhat really finished or so. Underlying math is the subject for rework since it’s not quite physically based at all.Publishing to start collaboration with other Cycles developers who are looking into solving this puzzle.
What do we consider a shadow catcher?

That’s a good question actually, and there’s no single formulation of what it exactly is and mathematically it’s a bit malformed in the constraints we’re working on. Ideally shadow catcher is a difference between image rendered without artificial objects and with them. Such approach gives best ever shadows, but takes 2x more time to render. So for good usability we need to get some assumptions, make system a bit more biased but give artists an useful tool.
Shadow catcher is mainly used by VFX artists to inject artificial objects into real footage. At least that definition we’ll stick to
in Blender. Hence here’s what shadow catcher should be capable of doing:

  • Receive shadows from other objects: be totally transparent when there’s no shadows cast on it, be more opaque in shaded areas.
  • Ignore self-shadowing and shading. Shadows caused by occlusion with itself already exists in the footage. Same applies to the

shading – all shading caused by material itself are also in the footage already.

  • Interact with other objects in the scene. This sounds a bit tricky but makes sense actually. Consider situation when one needs to put sharp glossy object into the footage: you’ll want objects from a real scene to be reflected in the artificial object. And often you’ll want the object on which shadow is to be cast to be reflected in such situations. Surely you can escape with copying object and playing with ray visibility, but that’s complicated scene setup instead of making it simpler.
  • Be affected with indirect light. Cycles is the GI render engine after all!

How to use the shadow catcher?

  • Create an object on which you want to receive shadow.
  • Create some basic material setup which is close to a real object.
  • Enable “Shadow Catcher” in Object buttons -> Cycles Settings.
  • Be happy! (hopefully, once we’ve debugged all the code)

What this patch actually contains?

It contains all the bits which tries to implement definition of shadow catcher above. It is trying to implement it all in a way so we don’t need to make big changes in the ray integration loop, hence it has some tricky magic to deduct what was the received shadow from the light passes and will fail in certain situations, mainly when there is no direct lighting of the object at all. It is totally tweakable to become more artists friendly, i just didn’t have enough time to try all the ideas and used whatever latest semi-working formula was.
Major changes are in fact made around shadow_blocked() to exclude shading from self. This part is based on an older patch which tried to expose it to an user. That exposing settings are somewhat malformed and shouldn’t really be used. In fact, we should remove those settings from the interface.

Cheers,
~Tung

thanks for the build, been waiting for this and it works nicely! but you always need to do some compositing -no problem with that- as just a model on a SC plane + an equirectangular won’t do it; but great to see this finally getting into cycles

Thanks for getting this going, been waiting for this for a lot of cg with live footage integration.

Comments so far form others are overall the standard compositor method is more usable but this works for a quick fix. Suggested improvements are more accuracy for shadow compositing, add catching reflections and increase precision in general.

Note: this is again in comparison to manual compositor methods in blender and shadow catchers from other render engines.

Great!!!

For my part it can immediately in the trunk. :yes:

there used to be an experimental shadow catcher made by lucas; does this one behave different ?. (better?)

I get different shadow tone using this shadow catcher than doing the shadows in the compositing. For a solid workflow it should be consistent. The shadows using this shadow catcher are usually darker than the original. Also it would be handy to catch also reflections.

New update coming this weekend to add tools to the compositor.

Is that link something related to this shadow catch patch we are discussing here? doesn’t looks like. The shadow catcher works on real-time viewport and also on the final render, the idea is to avoid using compositing nodes for this task.

Ahh, I didn’t realise this was real time. Although, mine takes the legwork out of the compositing nodes by doing it automatically, but yes, different thing. Sorry/


my test file : https://dl.dropboxusercontent.com/u/54767531/shadow%20catcher.blend

Using windows 10, got above strange result
i used a background image for grass photo, and a flat square as shadow catcher, i was curious what would happen with different light colors.

Also, note i think its better as a cycles nodes, and in such node i think it should be possible to give the shadow some color tint.
Because if one wants to combine blender with real movies or photo’s, the daylight temperature (in K) effects the shadow collor as well as camera filters, and artifical light (xenon,argon, TL, Led light etc)…

It will be nice to have a linux build please?

Sergey just updated the patch to current master according to what he said in IRC.

Hi.
I have not much idea about how shadow catcher should work. I have a couple of questions:

  1. Does it matter the material set on the object with Shadow Catcher? Or even should be no difference if the object does not have a material?

  2. Should the intensity of the shadows be similar comparing the shadow obtained on a plane with diffuse material (shadow catcher=off), and those obtained in the plane with shadow catcher?

Updated to a new build in the first post.

thanks, self shadowing is ok now, but a GPU sun still fails to cast shadows on not shadowcatcher objects -at least for me-
realtime previewing on images and tracked videos is great, but I miss a way to preview directly on world background though

There is a new implementation of the patch that seems to be working better. For now only with Branched Path Tracing.
Thanks Sergey!

Updated to the latest patch on first post.

Hi.
Sergey is also requesting to collect “real use cases” Shadow Catcher scenes (.blend files). So VFX artists could collaborate and help a lot providing such scenes and testing.