This has to do with how alpha actually works. In the Image Editor window that you’re seeing the results through, it doesn’t provide you with all the information.
The screenshot below shows the RGBA button, which is described in the documentation.
https://docs.blender.org/manual/en/dev/editors/uv_image/image/introduction.html#header
The indicated button, as the documentation states, replaces transparent pixels with the checkerboard background. However, that doesn’t mean that the pixels that are transparent don’t have any values. To view all your pixel info, click the RGB button to the right.
In the image below, I had 2 render layers. The first had a grid, the second a sphere. I used the sphere to set the alpha of the grid. Both images show what the composite looks like after the ‘set alpha’ node. The view on the left is with the viewer control set to RGB, and the one on the right is set to RGBA.
As you can see, none of the image info in the ‘transparent’ parts of the image is lost. This is the correct way to handle alpha, but the RGBA viewer setting is obviously misleading.
Since this is not your intended result, you need to multiply the RGB values by the alpha channel. One way to do this is on the first ‘alpha over’ node (where you have combined your rendered elements) check the ‘convert premul’ box.
If any of this wasn’t clear, don’t be afraid to ask more questions. And if you want to look into this whole alpha thing, here’s a good place to start.
https://wiki.blender.org/index.php/User:Sobotka/Alpha/So_WTF_is_Alpha