Render different part of a scene and then compose togheter

i tried to summarize all in the title.

recently i started a project modeling a human face.
the main issue are hairs.
since cycles has some issue rendering them, i swapped to yafaray. but it is at a really low stage on the mac version development and final output is horrible,

i then taught to split the render in two pahses: body in cycles and hairs in yafaray.

the main question is: is there any way to render those part separately, but keeping main shapes?
let’s say the body has a POSE and from camera part of the face is hidden, once i render separately, i would need the face to be partially covered when rendering it in yafaray, in order to compose them correctly…

is this approach correct, or is there something better?


I would render them both out and use and object mask in blender to combine them

interesting, it sounds like the trick i was looking for
can u give me more details pls?

but no other replies :frowning:
googling din’d produce anything interesting :frowning:
couldn’t find any info

Much depends on the details in your scene, but a general approach would be:

  1. decide which portions of the scene are to be rendered in Cycles and which in yafaray. You’ve probabaly already done this, so lets narrow it down to just a head and it’s hair for simplicity’s sake in this explanation.

  2. Create two scenes (different files are OK for this), the first for rendering all the Cycles elements, the second for all the yafaray elements

  3. Render the Cycles scene as an image sequence (PNG or OpenEXR recommended depending on your needs). If any of the elements in this portion will overlap those in the yafaray portion, you’ll need to use some kind of masking process for later compositing. This can be as simple as rendering with no BG and setting the file to Premultiply (for an Alpha Over composite), if Cycles supports that. Or some sort of keying method like blue- or green-screen might be appropriate.

  4. Render the yafaray BG elements (if any) as an image sequence (format dependent on use). This would likely have just normal lighting.

5a) Render the yafaray hair element (let’s call it the “Hair Plate”). Because the hair will both overlap and be obscured by parts of the Cycles imagery (mainly the head), a simple approach to rendering just the hair isn’t possible – parts of the scene will have to be made self-masking. There are a couple of ways to do this. If the hair is light-colored and never gets very dark due to shadows or the like, you can give everything else in the scene a matte black material and then render with the normal lighting. Alternately, everything but the hair can be given a specific color like a blue- or green-screen material, or even pure gray or white. The Shadeless Material property is useful for this approach. Which to use is highly dependent on the nature of the hair and the other elements. Regardless of the method, render this out to an image sequence as well.

5b) An alternative to self-masking all but the hair is to self-mask the hair itself. In this case, two rendering passes of the yafaray file will be needed, one with the hair normally lit and with normal materials, and one with the hair given special materials and perhaps special lighting. The first pass records the hair as you want it to look in the final composite, the second records the self-mask used to put it together with the Cycles imagery. In the second, the hair is given a material that allows it to be isolated in the Compositor from the rest of the scene, so depending on the rest of the scene elements, that could be matte black or white, or some keying color like a blue- or green-screen hue.

  1. In the Compositor, the rendered elements are set up each with an Input>Image node. The Cycles elements are first set up to overlay the BG elements using Alpha Over. The Hair Plate (regardless of whether 5a or 5b is used) is set up to be the next step in the composite, with appropriate Matte nodes (or something equivalent) that isolate the hair and let it be combined with the imagery from the earlier part of the “noodle.”

The basic premise is to render all parts that need to be isolated from one another separately, then use various masking and or matte methods to recombine them in the Compositor. Self-masking is the most efficient way to do this with elements that both overlap and are hidden by other parts of the scene at the same time, which hair usually is. i started a project modeling a human face.

you explanation is deep, complete and cool, but unluckily i couldn’t follow you almost at all :frowning:
i couldn’t catch the workflow at the base :frowning:

isn’t there any tutorial or a guide around to have a first look at?
nothing “Pro”, just a starting point to came here later on, once read :slight_smile:

first of all i can’t make out the “self-masking” concept.

By self masking I think he means rendering with RGB+alpha, this also means selecting a file type that supports an alpha channel and a background that shows a key. That is no objects that would clutter your clean alpha edges around the hair.

Actually by self-masking I mean that the masking needed for compositing is made part of the rendering rather than being generated by some other process. Think about it – a head of hair cannot be simply rendered by itself with alpha and then composited, because you have to mask out the part of the hair that’s obscured by the head (let’s say the back). But this same hair will also overlap the head in front, such as bangs, and external masking for the head will obscure this as well. Having both obscured and overlapping parts means you can’t use simple alpha-masking.

Instead, imagine the head rendered in place with the hair. Then, no matter the view, the head will obscure the proper parts of the hair, and the hair will overlap the proper parts of the head. So now the problem is how do you remove the head portion and leave just the “self-masked” hair? One way is to use some form of keying like chroma key. If the BG and the head are exactly the same color, and this color is keyed out during the compositing, what is left is the hair, already masked by the now keyed-out head, ready to be composited on top of the head that was rendered separately using another process (in this case, Cycles).

The choice of the matte process used – chroma, luminance, or whatever – would be somewhat dependent on the specific nature of the components used for the composite, but the self-masking technique could be used in most of them.

The self-masking method has been around for a long time, from the days when the first computer-controlled cameras allowed making identical passes on model shots – Douglass Trumbull used it on many Blade Runner shots – and just needs to be adapted to all-digital workflow.

The critical factor in any matting/compositing process is a perfect match between the composited elements, so you’d have to make sure that all the camera settings in both the Cycles and the yafaray renderings result in a perfect match between the separately-rendered parts, in terms of field of view, perspective, depth of field, etc. This may present more difficulties than the actual matte/composite process.

you are a true teacher :smiley:

let me check if i made a little step further: let’s say hair + face + body

cycles should render face + body + green* hair
yafaray should render hair + green* face + green* body

where green* should be a plain full (0 255 0) green color to composite it
so, in this way, elements would merge in terms of DoF, POV, shadows, but simply using compositing, they should be overlap correctly?

setting green could it just be assigning a diffuse material with no other properties?
and if i am correct, which nodes should i use to complete such overlapping?

You probably won’t need the green hair in the cycles render. If the hair is casting shadows, you can set it up with a material that ONLY casts shadows, but the strands don’t render. This way, you’ll have no possibility of problems with interactions between the strands in the different renderings. The hair need only be fully rendered in yafaray.

For the green setting, using the Shadeless property would give the pure color with no variation due to lighting, which is usually the best approach.

Other options instead of green would be using solid matte black or matte white, but a lot depends on the nature of your hair material. One consideration is transparency, which complicates the key color filtration process. In some cases you can’t filter the key color (e.g., the green) without reducing the hair to almost nothing. In that case it may be necessary to make a second matte pass on the hair, in a way to make it completely or nearly opaque, for use as a masking element in the composite. A separate material for the hair could do this.

Depending on your scene setup, you may be able to get by with using a keying color only for the head, and leave the BG empty of objects. If you set the Render>Shading option to Alpha: Premultiplied, then the Hair’s alpha could be used for compositing for all but the area occupied by the head. THis could simplify the filtering process in the Compositor.

The elements should fit properly IF (and this may be a big if) the difference between camera implementations in the different renderers is not too pronounced. There’s really no guarantee that the Cycles camera will perfectly match the yafaray camera, since they may render differently in terms of settings, focal effects, etc. So it’s important to test this before trying an animation – even a single frame would be helpful, but it would be better to test a number of frames with different camera framings, like far shots and closeups and in-between.

problem 1: cycles doesn’t support particles AT ALL. this means there’s no chance i can interact with them at render times
if i render in cycles i would have a full render with bald head.

may a solution be to render all BUT the complete head in cycles and the compose it later on, once i have full head yafaray rendered?

Given the “no hair in Cycles” limitation, my next question is “Why use Cycles?” Yeah, it’s all new and cool and has some groovy features and all, but if it has you jumping through ginourmous hoops to get your hair in the imagery, it seems like more work than it’s worth. In any case I think you’ll not get a good enough match in the way the head is rendered in yafaray to match the rest of the body and whatever else is done in Cycles. I’m pretty sure the compositing would be obvious.

I have no idea whether the style of your project will allow for no shadows from the hair. Probably not. But shadows can be composited in also. Just as you can create a scene for generating the hair plate (by “plate” i mean the images of the hair to be used in the composite), so can you create a scene for generating the shadow the hair creates on the head. But at this point things are getting very complex and again, it may be time to stand back and ask “Is this really the best approach to this problem?”

i am using both cycles and yafa as i don’t know luxrender very well and can’t reproduce the material the way i want

cycles if the fastest one and reproduces materials in a great way.
need to render hair properly, though, or the entire test is a fail :frowning:

just discovered that since cycles uses GI i can’t assign a plain matte material to the head and that for some strange reason doesn’t export alpha as well…
i am cmpletely stuck :frowning:

in any case i have to thank you for your patience, but i DO NOT have any masking/compositing experience and you keep speaking too difficult and i can’t follow you :frowning:

No problem, just ask for clarification. On the flip side, I have no Cycles experience yet, so I have to make some assumptions that may not be valid. After I get finished with some ongoing tasks and start rendering my demo reel frames, I’ll cobble together an example of how to do the stuff we’ve been discussing. Maybe that’ll be better than verbal descriptions.

As I mentioned at the outset, though, getting a good match between rendering engines may be the biggest hurdle, because if the hair rendered in yafaray doesn’t look like it was rendered in Cycles, for whatever reason, then the composite will be obvious and will likely fail for that reason, in terms of the overall look. It’s an ambitious approach to a not-so-easy-to-solve problem that stems from using a not-quite-mature rendering engine – Cycles.

i am now back with two screens, which should correspond to two different steps.

  1. giving green face produces green reflections on shoulders and makes it impossible to proceed.
  2. no alpha is exported. can’t remember if in some old test it worked and it is an actual mistake in the workflow (have at least 50 different test in the main folder ^^)
  3. yafaray export is a bit better: i have some green reflection, don’t know if there will be any setting to make the material completely non-reflective
  4. i have some issues with hair alpha, but you mentioned this in a early reply and i suppose there’s a solution for that.

i think this could be a good starting point to compare each other: i am not a pro and the model wouldn’t be the “modeler masterpiece”, but as an amateur and a 3D lover, i would like to learn something new, rather than box this project and start something different, just because this one is presenting unexpected issues :slight_smile:


I can see my explanations haven’t been thorough enough given your lack of experience in compositing, and for that I apologize. It’s hard to know how basic to get when discussing stuff in a forum, and hard to know what another poster may have for applicable background knowledge.

One of the main probs I see with your current efforts is that you’re including too much in the scene set up for compositing, that uses the green material. Here’s the basic breakdown of the plates used for compositing (again, “plates” refers to the separately rendered images or image sequences that will be used in the Compositor to assemble the final image):

  1. Figure fully rendered with normal lighting in Cycles, but no hair. This is the “beauty” plate for the figure. It includes the head, and can include whatever scene elements appear behind the hair or do not interact with it at all in the composition of the frame.
  2. Hair fully rendered in yafaray, normal lighting, but with the head included, but ONLY the head, or only the head plus whatever portion of the rest of the figure the hair might interact with, e.g., if the hair is long enough to touch the shoulders, include the shoulders. The hair is given its normal material and is lit normally. The head/head & shoulders model is given the keying color material – you’re using green here but other colors could also be used – and set up for no shading. If yafaray allows it, use the Shadeless option in the Material>Shading section. If not, then special lighting could be used that is restricted to only the layer with the green model parts. The idea is to have the model color render as completely flat, no shading at all, as much as possible. This is the beauty plate for the hair, plus self masking from the head model.
  3. As an option that may help in the Compositor, render the hair as in 2 but with a completely white material, and make the head pure flat black. This is the hair matte plate, intended for use only in the compositor as a masking element when needed.

The plan is that the hair, which can be isolated from the key color in the Compositor, will be overlayed on the figure beauty plate, making it appear as if the hair was rendered in place. By keeping the keying color only on the hair plate(s), you avoid all the problems with reflections that you’ve described. The #3 plate can be used in the Compositor to help reinforce any masking that may be needed.

Rather than go into more long description, I’m assembling a file that I can use to post example pics from. Not sure when it’ll be done, but soon as I can manage.

Here’s the process in pictures:

This is the target, what it should look like after compositing:
In this image the figure & hair were all rendered together in one pass

For the composite version, these are the “Figure Beauty” plate and the “Hair Beauty with Head Self-Masked” plate:

The Compositor Noodle:

The Figure Beauty Plate is just a straight rendering with normal lights.

The Hair Beauty with Head Self-Masked plate was made by giving the figure the Shadeless green material and a Mask modifier to limit the rendering to only the needed parts, the head & shoulders area. The hair has a normal material & lighting. The image was rendered to a PNG format with RGBA enabled so the alpha channel could also be used in compositing. The noodle is self-explanatory. I used a Channel Key node for the principle masking task, it seemed to work cleaner than either Chroma or Color Key. The Color Spill node cleans up the residue of green in the partially-transparent edges of the hairline. Comparing the Composite to the rendered-in-place Target image, it looks pretty good to me.

Though I used BI for all of these, the same approach should work with renderings from any engine, be it Cycles or yafaray.

Note that I did not use any shadows for the hair in this example, so it looks a bit odd. But Shadows can be rendered separately and composited in as well.

ok thanks to your guide i am starting clarifying all this issue…

but something went wrong…

but it seems i am doing some steps further.

one big issue remains the alpha of the hair…

in yafaray they look almost transparent. i don’t know if it is just a matter of material though :stuck_out_tongue: