Hey ! Thanks a lot for writing all that, it’s helps a lot to put things in context.
I’m not sure about the whole duration it’s ~20mn ? and it’s 13 scenes each containing multiple shots ?
Anyway, It’s indeed a big project
Yeah in fact it’s more emissive light from the character ? Rather than indirect light ?
In fact yeah light groups should give you what you need for emission. For shadows it’s a bit trickier as for cycles shadow is a byproduct of light and it’s hard to have only shadow without light in general.
Just an idea out of the blue I didn’t tried these tools so I might be wrong. Maybe it’s possible to use light-linking (or rather shadow linking) to produce a shadow without additional light.
Then with light groups you might be able to have all the different light except the one that is shadow only.
The idea is to have the beauty that include that shadow, and combining all the light groups passes to get a beauty without shadow. Then it should be possible to extract only the shadow by subtracting one with another.
Hum, not sure if you need to have a clay material, you just need to be able to add the light on top of your render, and multiply the shadow ?
Anyway, I don’t have much time now, but I’ll go back at it, maybe try something with how to extract these light and shadow pass from one render layer. It looks like a fun challenge.
the clay material is needed for the emissive from the characters, that’s because if I also have shaders and textures from the environment, when I add the pass on top of the beauty things will get a bit messy I guess… or am I wrong?
I managed to have the emissive light and beauty from a single render layer using light groups. However another problem came out. The light groups passes are not denoised. Now instead of being able to use the OIDN gpu denoser, which is really fast, I need to denoise 2 passes in the compositor, which is actually slower than rendering 2 render layers
after effects was picked by the client. It’s also the compositing program I have more experience, but I won’t lie, often I hate it At least they made some progress with handling linear color space workflow however
Hum, I think it should work just like with the light groups, so the pass should look like if you only have that light on your environment with their own textures and materials.
The result is the contribution of that light to the scene that you can add on top of other lights contributions.
If we pretend that for some reason the light is white, the set is red, and the clay is blue. Rendering a pass of the light + clay will result in something blue, that has nothing to do with the set. So instinctively I think you shouldn’t use a clay render if you want to keep things kind-of physically accurate.
For shadows things might get a bit different though. I think it’s either possible to extract a shadow pass using shadow linking + light groups.
Or if that turns out impossible to do, you’ll need something that looks like a shadow pass, that is a white background with a black shadow. So closer to a clay render. Or maybe, since the point of the shadow extraction is to do everything with one renderlayer, in that case you can just simply use shadow catcher… As you’ll need some renderlayers anyway …
Hum ok, I guess it’s because of the 6k renders And you can’t use the GPU compositor because you have renderlayers.
Did you make sure the renders are fine once animated ? Because many times I had surprises with denoised images that looks great but where blotchy once animated.
The best way to manage your complex scenes in Blender is to simply organize them better. And develop asset management strategies.
I would not recommend opening the can of worms that is using another format.
I watch it these initiatives even at the “Studio Level” with no small amount of skepticism.
Depending on your situation, it may be you are trying to do too much.
I can’t say. But you should not discount the strategy to simplify your pipeline or break it into manageable segments and not try to manage everything in one file, which is what linking and collections can do quite well if you learn to work with them well.
In our studio we usually have a tiered approach to the pipeline.
Well, that depends but it’s more what you should use for rendering , compositing and grading. Even though for grading you can probably have 16bits depths, and remove AOVs, and everything that is useful only for compositing.
However when editing you shouldn’t bring all that complexity to your NLE.
Most of the time the workflow is like this :
1/ storyboard / filming
2/ editing
3/ 3D
4/ Compositing
5/ Editing
6/ Grading
7/ Export
Proxy is just a mean to have realtime playback in the NLE.
If you really need to use the same EXR you’re using for compositing inside the NLE, then you should be able to do the editing work with proxy, and the compositing work with original EXR files.
And in the same way, when previewing your compositing work inside Nuke or else, you’ll use a lower resolution, like 1k or 2k instead of 6k in your case. Just so you can preview faster and get a responsive playback.
IIRC Nuke also allows to use a proxy version for the footage, so it computes even faster, but at this point I suspect that some pipeline scripts are needed to convert the renders and manage the proxies…
Why have you considered that using proxy would undermine an Aces workflow or impede your overall quality workflow?
Just a note. This has been true since nearly the beginning of NLE time.
The latest in standard resolution has always surpassed the ability of hardware playback.
With the exception of systems tuned to playback video with high speed drives and some kind of special hardware/firmware to process the video.
In our studio we use EXR. But I never edit with them. In fact I don’t edit with image sequences I have the render team render out .mp4 first.(from the exr sequence)
Then once all is good, a final pass is to load up all of the files again as EXR sequences to do the final grading and render to whatever the target video stream is. Usually some kind of lossless or uncompressed video.
From that usually then, the other delivery formats are generated.
EXR is not only expensive in space, it is expensive in render times compared to say, a png sequence.
When you have your final pass, you should only be needing to spot check sections of the video by rendering it out to a format you can playback.
Once it is all good, render the whole thing and wait.
But at this stage of the process, you are not requiring real time feedback.
I think you’re right, but I just can’t afford the cloud disk space required to host hundreds of thousands of frames in PIZ. Beside I am not sure AE would be able to play them in real time anyway. AE sure can’t play DWAB frames in real time, I am not sure anything can. ACtually, when you start adding effects and such, the AE render times are much higher than the cycles ones
I think as the others said below, the only way is to use proxies for when you need a snappy playback.
BTW, as far as I am concerned, this can sure be a general topic about pipeline, so feel free to ask this kind of questions
Another question, related to exporting multiple passes from the same render layer. For now I failed to export beauty, emission light on the environment from the characters and shadow pass using a single render layer.
I just managed to include the emission light pass in the same render layer as the beauty, but the fact that the light group is not denoised + the excruciating slow speed of the denoiser in the compositor made this achievement useless in terms of speed.
As for the shadow pass, I failed altogether.
I don’t know much about other render engines, so my question is: in other render engines (octane, redshift, vray, arnold, renderman, karma… whatever) is there a more efficient way to achieve this result? Is there a way to export these 3 passes efficiently in the same render pass with a gain in render times?
@Bernardo I just came back to respond to a user who commented on a tool I made to help address the rendering, and AOV side of what you’re talking about. I’m creating a kids tv show and one episode I think has 30 odd shots so I built a node based output and rendering panel for Blender. it’s not houdini Solaris for assembling at all but more of a Gaffer/Katana way of approaching this rendering/outputting of multiple layers. Making it easy to do matting and light groups. Thought i’d drop it here in case it would be any help to you.
Wow this looks amazing, thank you for sharing! I had a similar idea but I don’t have the coding skills to out something like this together, it’s great someone created it, I’ll check it out asap
I am taking a break from work at the moment. Yesterday I had a quick look, but certainly not enough to come back with some serious feedback.
I have a question however. How do you “build” a render layer? I saw you can add a collection, is there a way to merge more collections together? In my first quick test I wasn’t sure how the selection of collections I want to use for a certain render layer exactly works.
However the general idea is exactly what I had in mind, so it’s extremely promising!
Render Layers is the name of a compositing node referring to a ViewLayer.
A ViewLayer is a set of activations of collections. ViewLayers are created per scene, proper to a scene.
Active ViewLayer in UI is indicated in Topbar, next to Scene name.
By default, all Scene’s Collections are activated in ViewLayer.
User decides to exclude some from Viewlayer in Outliner, by ticking associated checkbox, or pressing E key.
To be visible into a ViewLayer, collections of objects have to be linked to its Scene.
Objects can be moved from one collection to another by drag and drop in outliner, or by using M key.
Collections can be nested by drag and drop, or by selecting parent collection before creating new one.
If you select only a collection item line and delete it, objects are not deleted. They are moved to parent collection.
Properties of ViewLayer (Passes, AOVs, Lightgroups, Filters, Overrides) are set in ViewLayer tab of Properties Editor.
