Render with background image

I want to render a 3D model on a background image and I also use shadow catcher on a ground plane. It all looks perfect when I do a simple render in the viewport (Shift + Z), but when I do a real render with the node setup as in the image below, the background image seems to be over exposed with to much light!

Is it the light from HDRI-image that affect the background? If so, is there a way to prevent the light from affecting the background image? Or are there other better options to handle this?

First of all, I would get a viewer node coming out of that Alpha Over node … then if you tick Backdrop and Auto Render at the bottom of the node panel…

Then… alter the premul slider in the alpha over node.