Solved camera shot EXTENDED in 3D

Imagine this: you film a subject in front of green screen with a handheld cam, there is a bit of shake. You solve the shot, and create a 3D world around it yada yada. Now, you want to ANIMATE over the camera to create a medium close up shot that quickly pulls out, making the subject tiny and far away. Obviously the issue here is that you would like to extend the camera move in 3D, while still keeping the camera solve. It’s no problem to animate the camera with a parented empty or something but I believe 2 problems arise:

  1. The footage you are used (let’s say a keyed out shot as a background image) doesn’t scale/move with the newly animated (extended) shot. I’ve done this effect once but I had to manually re-scale and re-position my subject in After Effects. It turned out okay but I feel as if there’s a better way to tackle the problem.

  2. If the extension or re-animation you did happened to be a pull away shot there would have to be some math done to match the solve’s movement with the new scaling of the animated shot. Example: without alteration or animation let’s say the camera shifted the subject 50px to the right over 1 frame. If you were to animate the 3d camera with an empty and pull back this newly animated camera so far that it made the footage 1/8th of it’s original size (assuming we’ve solved this in problem #1) the parented camera solve still moves the footage 50px to the right over 1 frame, clearly this will not work. It would now need to adjust and move the footage about 6.25px (50px*(1/8 scale)) for the extension animation to look correct.

I feel as if this isn’t the most complex thing in the world to pull off and I’m sure with a bit of coding or parenting it could be done, does anyone have any solutions, thoughts or ideas about this problem? I do a ton of VFX with blender where shot extensions like or similar to what I have described would be really helpful.

Thanks for reading through and putting up with my poorly worded description