Cloth gets attracted to collision objects [a different issue from May 2020 post]

Hello, blenderheads,

Have I got ‘brain hernia’ already with this:

Blender 2.91. The figure is in fact animated, goes from a T-pose to the current one over 80 frames.

Book and stick placeholders are parented to respective hands’ bones. Quality settings for both cloth and collision are 8/8 respectively (they were 2-4 times higher, which was a hell of time rebaking).

Cloth seems to get magnetic arbitrary and goes attracted to the rod as in the image attached, falls out and then sticks back again. No such issues with the ‘book’. Scale applied, topology is all the way quad only, and everything is really big in BU/meters, while the distance settings are minuscule. What can be done about it?