Mask(solved)

Support Compositing and Post Processing
1

The cause of the problem was nicely pointed out here, it was the render settings.

2

Blend is set to cross instead of alpha over.