Making verts follow (mocap) tracks

Hi all.
I’ve tracked a video of my arm, and created a model arm that I want to perfectly follow the tracks.
I tried applying an object solver, but as my wrist rotates the mesh doesn’t follow properly. I mitigated this by using AnimAll to animate the vertices on top of the object solver, but it’s proving very (very) difficult to use accurately in conjunction with the solver.
So I tried a new approach - hooking verts to empties, and applying Follow Track to these empties. This would work great, except with my arm rotating, all of the tracks get hidden at some point, meaning the empties and therefore the verts just stop moving until they get picked up again by the tracks.
My question is, how can I either momentarily disable the Follow Track modifier on the empties, or is there a much easier way to make verts/vert groups follow tracks? I’ve even tried using armatures with constraints, but I can’t get it to work.
I’ve searched forums and Google to no avail, so any help would be greatly appreciated.
Many thanks in advance.