Well, a couple of terms that will put you in the right direction are: matchmoving & camera-tracking.
You seem to be at the heart of the problem already! This is good, this is very good.
Trying to manually match the camera movement would be tedious at the best of times and very difficult/impossible for the majority of the time.
There’s a pair of free programs that can solve the motion of the camera, they’re Icarus & Voodoo. After that, there’s a whole bunch of commercial ones. SynthEyes, PFTrack, BouJou etc. (BouJou is a great one…) You get tutes on how to use them provided.
Yup, unfortunately, you do. I wrote a small tute on sticking a blender animation ontop of a photo. This explains the use of render layers needed to create CG shadows to stick onto your real image/footage. You can find it here: http://www.savefile.com/projects/808555944
As for books, a pretty good one is: Sybex.Inc.Matchmoving.The.Invisible.Art.of.Camera.Tracking.ISBN0782144039.pdf
There’s also competitions around for matching animation to footage that you have no know measurements of. I assume there’d be some kind of learning-oriented discussion to be found somewhere around them.
Oh yeah nearly forgot!! – take a look around for BLenses - a script to help with lens matching between blender and the real world.
- Have a search for posts on this board by a member called aussie. He’s written a tute on the subject too.