You can use the light path node to replace what hdri the camera sees:
This setup will make any background shader that’s in the second slot of the mix shader what the camera sees. This will also allow you to use a completely different image from what is seen by the camera to light the scene.