MIDI driven animations


Some time ago I’ve mentioned that I want to make a MIDI driven animation. To achieve this I’ve started from writing a script which will allow synchronising predefined, simple actions with desired notes from MIDI sequence. Here are the first testing results:
http://img714.imageshack.us/img714/3116/miditest01.jpg http://img208.imageshack.us/img208/8786/miditest02.jpg


I’ll post here more tests as the script functionalities will be expanded. Currently script is working only with Objects, but I’ve already started writing the part for Armatures. As soon as it will become more useful and less error prone I will try to publish it together with comprehensive manual.

UPDATE June 09, 2010
The last in the series of testing animations:

UPDATE August 23, 2010
At last, the script is available to your use. To download it, visit the MIDI Driver thread or go directly to the MIDI Driver homepage.


You are possibly the greatest hero ever and the world would owe you all their gold and frankincense if you release the kick arse MIDI blender deal thingamabob plugin script music video maker dream heaven on a stick.

SirGagarin, I think you are going in the right direction. I watched all of your demo videos. And any of the Animusic videos is just the best production to inspire you.

Several Blender Users have tried to do this, but it seems they gave up, including myself. I do not know how to write code in Python. Four years ago, I used a script written by jbperin to do this:


It is very simple.

I will be watching your progress.


Thanks for response. I didn’t expected anyone will be still interested with MIDI animations while this is “yet another” attempt to the topic.

loopduplicate: Thanks. I hope I’ll be able to fulfil your expectations.
okchoir: Nice vid. You’ve managed to create something good with the script I wasn’t even able to set up properly. BTW, I thought this script was considered “finished”. Well, I will definitely finish mine, because I made it my M.Sc. work topic. :wink:

Here’s some new test results. I’ve changed the script a little, so now it is also possible to animate lamps, materials, textures and world properties:

To whet your appetite, here’s a screenshot of the typical session with the script. Middle, blank part of script’s interface is left for MIDI structure display (I’m fed up with continuous jumping between Blender and sampler). Adding some selection tool there would also make the work easier.
Next time I’ll try to show something with armatures.


I like it! And although I cannot see evidence yet in your demos, I hope your goals include velocities, chords (simultaneous MIDI notes on the same MIDI channel), multiple MIDI channels, tempo changes, etc.

Surely, you have seen that Blender 2.5 can animate everything, someday.

very sweet :slight_smile:

This looks really good :slight_smile: Will there be options to customize attack and falloff. When the hammer moves around in test 3, its speed is completely determined by the distance between notes and mallets. It would look more natural, if it had roughly the same speed of movement always, and “waited when it had time”…

You have a truly fantastic idea going here, thank you for sharing what you’re working on. I was wondering, can your script also be tailored to move according to volume as well as notes? Ie a block that jumps in time to music and jumps higher on the louder notes?

I guess it should easy enough to read and generate animation from note velocity as well as note duration…

Really like this, I’m already having some ideas for this script, please do take the time to design a easy to use interface, it’s going to ensure you get to many people xD

Once again thanks for comments and useful suggestions.

okchoir: The tempo change is handled - note timing is calculated into milliseconds and can be easily recalculated into frames using current scene FPS settings. Thanks for mentioning velocity, I totally forgot about it. Now, current velocity value is used to scale the Y position of the keyframes to vary the intensity of the action respective to particular note (Marcion - exactly what you asked about).
Chords are quite difficult to handle but can be generated with this script using some tricks (I’ll present it when I find some suitable piece of music)

yoff: The idea here is to use Ipo or Action which defines animation for single occurrence of particular note or set of notes. This way user can define animations consisting of much more than attack and falloff profile. The task of the script is to insert this “elementary” Ipo into “final” Ipo each time the respective note occurs in MIDI file. Duration of the “elementary” Ipo can be of course scaled to the length of the note. Ipo is also shifted so that its selected keyframe falls on the “contact” point of the sound.
Thanks for pointing out the mallet issue - I’ve started resolving it.

cecilff: I try to do my best with the interface. Hope it’ll be comfortable not only for me. :slight_smile:

Getting back to the point: actions synchronisation is ready. Now, also the animation of armatures can be synchronised:

The animation was completed in a great rush, just to check if everything works, so it is a little bit messy. Now it’s time to refine the script, add some final improvements and start general testing in a bigger project.

Stay tuned for more MIDI-animations.
Oh, and happy Easter everyone!


And Happy Easter to you. This is amazing!

omg that is awesome, cant wait for the script

i like good animations very awesome animations

Love the robot animation :slight_smile:
So you have an action for each note? I noticed that middle and low rods were hit from the top while the high bars were poked :slight_smile:
How do you make it switch arms, then?

Very nice. Are you aware of the work Nick Porcaro is doing? See or www.porcaro.org Real-time visualizations with midi input.



Thanks for comments.
yoff: One can select every second, third etc. note with some offset (ex. starting from the first note for left hand and from the second note for right hand). This is what the Off option does (see the bottom of this post).
MikeCuffe: Thanks for link, I didn’t know about it. Really interesting work.

Let’s get back to animations.
Lately I came across this and decided to animate it in a Yamato-Stomp-Robot style:

Next, I found in my collection one interesting MIDI of unknown author and source. I’ve decided to make a MIDI band to play it. First I made the part of the drummer:

Then the the accordion and tuba joined in:
I’ve managed to apply two-note chords to the accordionist – handling chords with the script is tricky but possible.

Getting back to the development of the script, it has changed since the last time. Now it looks like that. The top part is the note filter where you can select notes by channel, instrument, value, time range and order (every second, third etc. note). Copy option allows you to restrict next selection to the currently selected notes. Middle is the graphical viewer of MIDI file structure with the possibility of a single note selection by click. The box selection is also possible. Bottom is the animation part: you can specify the type of the data-block, object/data-block name, Ipo/Action name and some additional options that would be long to describe here. You can synchronise up to nine actions at once.
Still, some heavy testing needs to be done along with tutorials. I think that July is the most possible release date.

Looking forward to your comments.

So is the script ready for release yet?

Your video test looks great. They really are more than tests.

Oh, WOW! You are making great progress.

Nice work!