Cloth simulation keeps intersecting despite having self-collision active

Strangely I couldn’t find a solution to this problem that seems occur every time I use cloth simulation.
I’m trying to wrap a fabric around a pole, at some point of the simulation both sides of the fabric intersect then the simulation goes to hell.
The pole has collision, and the fabric has self-collision, so it should NOT intersect. But it does, and once it does, the fabric scrunches and intersects both the pole and itself and shrinks into itself.

I tried activating/desactivating self collision, I tried tweaking the self collision parameters like friction and distance, the result is always the same.
Also, since no one else seems to have this problem as I couldn’t find another similar topic, or ones that are 6 years old, maybe I’m missing something.
But seriously I don’t get it.