Cloth simulation rigid vest

Hey ! I’m always struggling with the blender cloth simulation. I have a vest and it’s looking like that :


but when I run a simple cloth sim with self collision and collision enabled, I get that :
image
how is it possible to make the zones where there is red arrows, to be rigid and not collapse on themselves ?
Those are my parameters :