Leaves (Particles) glitch when mesh is animated

I have a scene with a basic tree that I created to line up with some real footage I shot. I used the tracker to track some points on the footage and then I used hooks to animate the tree limbs. So far, so good. My basic tree matches the movement of my footage.

Now, when I add a particle system to add leaves to my tree, the particles behave very erratically.

With the hooks disabled in the mod stack, the tree (my emitter) and the particle system play nicely. All of the forces I have set up work well and the animation of the falling leaves is perfect.

If I bake my simulation with the hooks turned on (the particle system is after the hooks in the modifier stack so the movement is taken into account) the particles glitch in and out, appearing randomly both along the tree and sometimes in empty space nearby.

I’ve tried baking with every combination of settings I can think of, from changing from faces to points, turning the modifiers on and off, having the “use modifier stack” turned on and off, etc, and it still always goes crazy.

I tried to create an armature rig for the tree instead of using a hook system and it was no better. It seems that once the mesh deforms the particle system isn’t happy. The tree geo is the most basic of basic models.

One workaround I have is to split the tree mesh into separate objects per hook influence and just have each of those with a particle system. That works and I’ll go down this path if I can’t find a better solution today. My main issue there is that I don’t want to have 20 separate particle systems for each part, which I would need if I want to use a vertex group to control my distribution, which is needed for artistic purposes. This file will also be passed to a colleague so keeping it as clean as possible is also important to me.

Has anybody seen this kind of behaviour before?