Image with trees should be rendered with with Alpha mode: Straight.
Nodes:
To mix this one I’d use regular Mix node (Mix blend mode). I’d use the image with trees as second input (lower one) and check “Include Alpha of second input…” option. Set Factor to 1.
Should work fine.
The problems arise from premultiplication. You shouldn’t have premultiplied the image when rendered.
This leads to RGB values that are premultiplied by alpha, so on the edges you have darker RGB.