Hi, I used the mask tool and mask node to create a mask in the compositor. I have checked both Anti-Alias and Feather check boxes, but they don’t seem to make any difference in the quality of the mask. It seems very stair-stepped (see image below). Am I missing something? Thanks very much.
Zaffer
From the image I can’t exactly say what causes the aliased alpha, but the comp setup itself is strange. Why do you have different scaling for mask and girl element? Do your masks on the size of the element you are rotoing, then inject the mask into it with Set Alpha node, do a Premult (Alpha Convert node) and over it with BG using AlphaOver node. If you need to scale the FG, apply scale after the premult.
HI Kesonmis, Thanks! I fixed the scaling problems by making both images and the render resolution all the same size (duh – should have done that in the first place). After that was fixed, the anti-alias on the mask node worked just fine. I did put in a Set Alpha node and Alpha Convert node, but they didn’t seem to make any difference. Thanks again for your help.
Zaffer
The set alpha and alpha convert go after the image, not mask. And mask goes into the alpha input of set alpha node. What they do is that set alpha adds alpha channel to image stream and alpha convert multiplies the RGB values with alpha, which results in so called premultiplied image. AlphaOver node expects premultiplied image by default and adding the premult makes the whole sequence of operations more explicit.
Thanks Kesonmis,
I’ll set it up that way.
Zaffer