Color value over 1.0

I’ve been struggling with nodes the whole evening, not understanding why my cluster of particles would not get fully transparent. I just realized, by dragging over the output, that colors are like “1.500” and more. It seems Alpha 0 won’t really be Alpha 0 in these circumstances.

The problem is, I’m trying to punch a fully transparent hole in the particles based on an Alpha Mask.
What I have : a renderlayer with a fully transparent background and the particles (with colors going over 1.0 most of the time). An alpha mask in black and white.
What I need : a fully transparent background + the particles - the mask.

How do I go about doing this efficiently?

Your question is a bit difficult to understand without a .blend but I can tell you this: the mix node is your best friend when it comes to masking.

If you run the problem alpha channel through a color ramp it will clamp clamp the RGB values of the alpha channel then either multiply the RGB channels by the result of the color ramp or you can use a set alpha node which wil yield the same result.