Chains of rigidbody doing crazy if parent goes fast

Hi folks !

RigidBody_wtf

What I’m trying to do here is to have chains of rigidbodies, linked together by empties acting as links, that would simulate strands of hair. (the idea is that the bones follow their rotation, so I get an animation with physics on mesh hairs).
The problem is that, now that the animation of the model is completely done, I’ve launched the simulation and…well, the rigidbodies go all around, looking anything but realistic. The links seem to break, moving around when they shouldn’t be able to because of localisation limit, and they pretty much get the hair equivalent of epilepsy. (I really don’t know how to describe it, which makes it pretty hard to find solutions)

So far it seems that the speed of their parent (the character rig and model) migth be the cause.
I’ve tried a lot of different things, and none has worked, any idea ?
Images detailing the process :


Here is the blend file : https://pasteall.org/media/4/b/4be97a06759b7480152fa1557a87e595.blend
Also I’m using Blender 2.83.2 currently.

Thanks !