Shadow catcher in Camera map scene

Hi there. I am trying to integrate models into a camera mapped scene and am having some issues using the shadow catcher in it. It is a pretty basic scene: Image of empty room, recreate the geometry of the room, project the image as the texture from camera view and set as emission. This lights all the models I put in the scene perfectly, except of course that it can’t produce shadows on the surrounding emission. But any plane that I add as a shadow catcher seems to catch shadows from the entire room - the whole thing just shades a dark grey. Any ideas on how to fix this or another method to get shadows from the scene would be greatly helpful.

Thanks,
Jon