I added GRAB_CURSOR and BLOCKING to my modal operator so the mouse cursor wraps around when I perform certain operations.
I also have a simple text UI that I made using blf.draw(…)
Since my UI is drawn using event.mouse_x and event.mouse_y my UI eventually gets drawn out of view because those values keep growing endlessly now.