Interesting question, I would say a game engine would be better suited for realtime performance. I am not aware of collision detection possibilities in Blender/EEVEE for instance, or getting easily data from physics. The later being probably not that obvious to do even with a game engine.
If your processing tends to be heavy the right way would be to use GLSL (which is C like language allowing to address the GPU directly) in the context of a game engine (they all allow that, and some like Godot have builtin OSC features).
EEVEE apparently doesn’t allow this directly but the shaders made with the node editor are (most probably) converted in GLSL internally.
The BGE was letting users to do some GLSL programming, I presume the UPBGE has preserved that well thought after feature.
To sum up, AddRoutes is not very well suited for VJ like performance, it can do that occasionally but has still to be considered as a preview facility in order to produce a non-interactive animation.
Sending from Blender using OSC/MIDI is currently less developed that the receiving side and the Blender python API allows to get properties values mostly. But I doubt it’s possible to get particles positions for instance. There is no real events system internally as you would find in a game engine or even in a web browser (actually I think there is something new emerging, I have to dig into that BTW but last time I checked the documentation was scarce).