I’m trying to have a 3d object come out from behind a object in a movie. So i masked the movie for the top layer but i dont know how to put the other layers underneath. is there a tutorial somewhere or could someone tell me the basic node setup. if you want the .blend i can upload it somewhere.
It all depends on your scene but typically you would stack the layers in the node editor from back to front - movie with object mask/object with FGmask/movie with neg object mask.
So import the movie footage. You will use the same footage for both the background and foreground live footage. Overlay the background footage with the object. A simple alpha over can work. Then alpha over the object with the foreground footage and the a reveal mask blocking out the portion of the object obscured by the foreground.
Let’s say it was a car coming out from behind a building. The first or bottom layer would be the location footage with the car alpha overed. Try putting the image of the object (car) in the lower image socket and the footage in the upper image socket. The sockets are arranged sort of backwards in that the lower socket is stacked on TOP of the upper socket. As the object is self masking we don’t need the factor socket in this case. Make sure that the alpha in the shading section of the render tab is set to transparent. So now, let’s call this AlphaOver1. Next would be the location footage on top of the output of AlphaOver1 with the reveal mask. Put the output of AlphaOver1 in the lower image socket, the footage in the upper image socket and the mask in the factor socket. depending on how you did the mask you may need to invert it first. Let’s call this AlphaOver2. There you go. That should be it.
Obviously I’m simplifying this as all the details are shot dependent. I used the alpha over as an example because it’s a great first try and simple for this answer. Try that and good luck. I hope this helps.
If the object is fairly simple, like a building, you can build a 3D model of it, put it in its own layer, and then set that layer as a Mask Layer. It will block out any part of the other object that is behind it.
[ATTACH=CONFIG]249558[/ATTACH][ATTACH=CONFIG]249558[/ATTACH]
Steve S
thanks guys, it worked.
A few projects ago, I had to whip up a “garbage matte” to get rid of a small rendering mistake. All that I did was to put an ordinary plane into a shot which used (linked to…) the same set and camera-angle. Yup, if you looked at its render there was a bright blue card sitting there, hanging in mid-air. But I didn’t use the visual data; I only needed it as an alpha-mask with z-depth. Problem solved. Perfectly? No. Acceptably? Yep. “Cheating a shot” is a noble occupation.
Only CG-geeks stare at a shot frame-by-frame closely. If George Lucas could ship “Star Wars Episode 1” with colored Q-tips substituting for a crowd in a cutaway of the pod-racer sequence, you can also get away with a fair amount of murder. 
Just be convincing “enough” and be sure that the audience is looking the other way.