If you look at my WIP from post 45 onwards, you can see I have already done all this in Animation Nodes, I am quite happy to discuss this with you direct by PM, or on this thread. I already have many generators, effects, recorders, sequencers, etc. etc some 60 nodes so far…
I am not an expert in Python by any means, but I can help you build AN Nodes, let’s talk! You will need a version of Blender after 6 May 2019 for the new Modulate() function that @neXyon has built into the AudaSpace API for me. For MIDI I use PyGame - I got on much better with this as opposed to RTmidi, I have a Korg nanoKontrol & a StudioLogic TMK88 keyboard connected to Blender and Reason through an iRig interface.
Cheers, Clock. 
PS. Thanks @skuax for the mention!