Cell fracture + rigid body glitch?

Hello,
I’ve a cave scene. I’m trying to get an effect of stalactite breaking and collapsing in the cave after an impact. My stalactite Cell fracture sim is perfectly working and falling like i want, but when i add a rigid body to the cave to simulate the collision, it make it just explode and weirdly react.
Do somebody know why it happens and how to fix it ?

Here’s video if you want to see the issue : https://drive.google.com/file/d/1nnBmQW6PrMEzZzTWGhw83YB_PxGcKh5V/view

Thanks by advance!