The first thing we need to do, is switch to Cycles, and create a holdout material.
At this point, our scene is looking really, really, boring… To see what the holdout shader can really do, we’ll need to enable transparent in the render settings. Note that this will also make your sky transparent (hence the green cube).
If your goal is to use this in the compositor, hit Render (F12), and you’re good to go. If you want the transparency in the final image (e.g. for Photoshop/GIMP), there’s one more step. Change the color mode (Render Settings -> Output), to RGBA, instead of the default RGB.
Let me know if you have any questions, or tutorial requests. If it’s a compositing problem, please post a screenshot of your node setup.