How would you replace an motion capture actor with a smaller CGI model?

For example in The first hobbit movie, they captured Andy Serkis acting as golum within a motion capture suit. However the model of golum is smaller than the actual actor, meaning if they overlayed the model ontop of the original footage it wouldn’t match up and you would still see the actor in the backplate.
How do big companies achieve this effect. Do they shoot the same scene 2 times for a backplate?
I’m just curious.


they have massively expensive motion control cranes for the movies. this is a computer controlled camera movement that is progammed to move the camera in exactly the same way over and over again so they film loads of footage with the actors, and then do a clean plate pass without any actors then they create a virtual camera pass in the cg software that matches the real motion controlled movements ( some (maybe all nowadays) motion control programs can take the camera movements from the rig and put it straight into the 3d software so no 3d tracking is required. then the 3d motion capture data is applied to the 3d model which is then adjusted by animators to fit perfectly and composited into the scene.