Page 1 of 2 12 LastLast
Results 1 to 20 of 25
  1. #1
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549

    Sound Bake & MIDI Sync Animations

    All done with Animation Nodes and a few base meshes, with Instances:



    I am now waiting for a MIDI Node that would read a MIDI file, create a series of controls with F-Curves derived from the MIDI notes and tracks so I can play instruments. I can do this with the old Blender 2.49 script, but this would be a great addition to AN.

    Here's a picture of two keyboards playing chords and melody using this method:

    Click image for larger version. 

Name:	Screen Shot 2017-10-02 at 08.43.14.png 
Views:	16 
Size:	666.2 KB 
ID:	500436

    I may render this, it's the first guitar solo from Comfortably Numb - I played this on my TMK88 connected through an iRig into my iMac and input into Reason DAW.

    Cheers, Clock.
    Last edited by clockmender; 03-Oct-17 at 14:08.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  2. #2
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    Update!

    I have now written a Python script that reads a MIDI CSV File - yes for now I have to convert the file to CSV format and there is a very nice little programme on the 'net to do that....

    The script examines the MIDI file and creates an empty for each note played in each track and keyframes the movement of the empty in line with when the script sees "note on" and "note off" events in the MIDI file I have tested it quite a lot, but it is still a prototype and really needs to be built into AN, or the UI, so the user can select the CSV file from their machine. I am currently testing my AN MIDI node tree on this new script's output to make sure it all hangs together.

    If anyone can point me in the right direction to build it into the UI I should be most grateful!

    Here's a screenshot of the control empties from Procul Harem's "Homburg", what I played on my TMK88 though blah, blah, blah. The selected one is the Hi-Hat (I think...).

    Click image for larger version. 

Name:	Screen Shot 2017-10-03 at 17.00.46.png 
Views:	13 
Size:	318.3 KB 
ID:	500635

    and a bit of the MIDI CSV File:

    Click image for larger version. 

Name:	Screen Shot 2017-10-03 at 17.02.46.png 
Views:	7 
Size:	85.6 KB 
ID:	500636

    Cheers, Clock.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  3. #3
    Member
    Join Date
    Aug 2012
    Location
    Arizona
    Posts
    305
    Awesome work @clockmender I'm looking forward to see where this goes and hope that some musically inclined Blender artists join your efforts to bring a midi-sync signal to generative animation!!!
    Lost in the voltage of a Eurorack patch gone mad. http://www.johnwise.com



  4. #4
    Member
    Join Date
    Jun 2011
    Location
    Sydney, Australia
    Posts
    513
    Do you have the Python script that converts the MIDI into CSV? I want to give it a try with Sonic Pi 3 and AN!



  5. #5
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    Originally Posted by enzyme69 View Post
    Do you have the Python script that converts the MIDI into CSV? I want to give it a try with Sonic Pi 3 and AN!
    I think it is written in C++? you can find it here:

    http://www.fourmilab.ch/webtools/midicsv/

    It works well for me and produces a clean file that Python can read easily and quickly - being a CSV file you can easily see the structure of the events in the MIDI file. Writing a script to read the generic MIDI file will probably be a huge pain in the ass, whereas Python reading the CSV file is easy, with not very much code. My script made and keyframed a multi channel CSV file with 102 separate control empties, total song length 6350 frames at 24fps in under 2 seconds...

    Cheers, Clock.

    EDIT:

    @JWise - thanks for the kind comments sir! - hopefully there will be enough interest to get this project really moving.
    Last edited by clockmender; 04-Oct-17 at 11:34.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  6. #6
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    Update:

    OK, so I have now got the script to:

    1) Read and action the note velocity as well as just note-on, note-off.

    2) Read and action the Header information, Tempo, Channel Names and PPQN (Pulses per Quarter-Note). it makes up channel names if they do not have a channel "Title".

    3) Calculate Beats Per Minute (BPM) and bar length.

    4) Have a variable, user defined "easing" to slope the F-Curves so the piano keys (for example) don't just bang down,

    5) Take care of the situation where a note is played immediately after another play - so the timings for Note-off[event-1] and Note-on[event-2] may be the same - this caused Blender to mess up the F-curves, so I sorted that out....

    6) Create the Group names for the controls, so they can be fed straight into the AN Node Tree.

    7) Set the frames precisely so the end of the MIDI sequence is exactly in line with the Sound - well to the nearest 0.0001 or a frame anyway.

    I am going to keep testing with ever more complex midi files to see if I can break it. the nest step is to vary the tempo (and therefore BPM) throughout the MIDI file and action this in the animation. The PPQN won't change but the Pulse time per Quarter-Note will (130bpm = 461538 micro-seconds / quarter note as a "for instance") Any tempo changes are stored in the MIDI where they occur, so I should be able to read this and adjust the frame translation from the MIDI pulse value.

    Cheers, Clock.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  7. #7
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    Update:

    Opps - Youtube screwed up my video....

    I'll try again later.

    Cheers, Clock.
    Last edited by clockmender; 06-Oct-17 at 11:33.



  8. #8
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    OK, here is a video for review:



    Let me know what you think to this please.

    Cheers, Clock.
    Last edited by clockmender; 08-Oct-17 at 07:08.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  9. #9
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    Here's an example of a set of controls built by the script:

    Click image for larger version. 

Name:	Screen Shot 2017-10-08 at 18.25.06.png 
Views:	9 
Size:	377.4 KB 
ID:	501129

    I have changed the "Empty Draw Type" to "Single Arrow" and checked on the "Display Name" for each empty (all done in the script) You should also be able to see the Group Name - "Channel_2" for the selected empty. The MIDI file did not have Channel Names, so I add these in the script using "Channel_" plus the MIDI channel number, otherwise it takes it's name from the MIDI file.

    The top set are the drums, the "missing" lines are controllers, that we do not need to animate the keys. You can also see a section of the F-Curve for the selected control - this has near vertical sides as the notes are very short in the MIDI file, particularly for the drums. Longer notes means I can use a larger "easing" values to slope the sides of the F-Curve a little more. You can also see that the Keyframes are not on an integer frame number - you can do this in Python, but not in the UI, or should I say I haven't found a way to do it in the UI yet....

    So now I have got all the various bits working, now for some serious testing on many MIDI files to try and break it.

    Hope all this makes sense. I am going to try to build the script into an Animation Node, with Input boxes for all the parameters in the script - this I have never done before so it may take some time.

    Cheers, Clock.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  10. #10
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    I am really struggling to make a new node with my script in it...

    I think it is going to take a while to get my head around what is allowed and what causes Blender to kick AN into touch.

    Has anyone got any pointers, has anyone done this before? Help gratefully received.

    Cheers, Clock.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  11. #11
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    Eureka!!!!

    Click image for larger version. 

Name:	Screen Shot 2017-10-16 at 17.47.29.png 
Views:	7 
Size:	318.4 KB 
ID:	501984

    I have made a Node - that creates controls with F-Curves derived from a CSV format MIDI file, I am very pleased with myself.

    I will continue testing and error trapping before I declare it "Fit for peer testing", thence onto "Fit for purpose".

    Cheers, Clock.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  12. #12
    Member minoribus's Avatar
    Join Date
    Feb 2013
    Location
    Germany
    Posts
    3,304
    I think I said some time ago that you are doing wizardry here and I can only repeat that Awesome advanced stuff. Keep it up!



  13. #13
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    Why thank you - I am building a new node to calculate the indices as a one-off operation... may take some time as I haven't a clue what I am doing.

    Cheers, Clock.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  14. #14
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    Originally Posted by clockmender View Post
    Why thank you - I am building a new node to calculate the indices as a one-off operation... may take some time as I haven't a clue what I am doing.

    Cheers, Clock.
    OK, so now with a great deal of help from Jacques I have made this new node. To get it to only execute once in the project, I have put it in a new node tree and used Group Nodes to pass the index file from tree to tree:

    Click image for larger version. 

Name:	Screen Shot 2017-10-18 at 11.12.37.png 
Views:	2 
Size:	73.8 KB 
ID:	502115

    And the animation Node Tree (or part thereof):

    Click image for larger version. 

Name:	Screen Shot 2017-10-18 at 11.13.00.png 
Views:	2 
Size:	103.0 KB 
ID:	502116

    Cheers, Clock.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  15. #15
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549

    The "Rude Band" Plays for the First Time!

    So here is my latest project:

    Click image for larger version. 

Name:	rude-band-1.png 
Views:	3 
Size:	570.0 KB 
ID:	502129

    And a closer view:

    Click image for larger version. 

Name:	rude-band-2.png 
Views:	3 
Size:	816.7 KB 
ID:	502128

    They all "gesture" to the tune "The Entertainer" by Scott Joplin, each puppet represents a note on the piano and they are arranged in Octave Sets rising from the lower tier, note C on the right as they stand, B on the left. I must apologise to the late Mr. Joplin for this project!

    I wil render it to a video, once I can stop laughing and transfer it to my server for processing.

    Cheers, Clock.

    PS. Seriously - I can now animate almost anything using my new MIDI Nodes.
    Last edited by clockmender; 18-Oct-17 at 08:34.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  16. #16
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549

    Progress

    So now I have revised nodes and get a much better performance:

    Click image for larger version. 

Name:	Screen Shot 2017-10-25 at 18.20.56.png 
Views:	11 
Size:	174.0 KB 
ID:	502930

    Two different tunes in the same file, two setup groups and one execute group. Revised nodes work well, although I am currently working on a way to get rid of the control objects and store F-Curves in a new Class - wish me luck, this stuff isn't easy!

    Once happy with these nodes I will put them on GitHub, for peer testing, if you are interested that is

    Cheers, Clock.

    EDIT:

    I forgot to say this node tree runs at 1.2ms - 149 notes played over 5760 frames....
    Last edited by clockmender; 25-Oct-17 at 12:27.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  17. #17
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549

    Yet More Progress!!!!

    Here are three manuals playing Toccata & Fugue:

    Click image for larger version. 

Name:	Screen Shot 2017-10-25 at 20.52.51.png 
Views:	12 
Size:	416.5 KB 
ID:	502949

    So why have I bothered to show this? You may well ask and the answer is because this piece has hundreds of tempo changes and several key signature changes in it. I have made some experimental code, that I am not prepared to post on GitHub yet, that takes all these tempo changes into account. There are some errors that I still need to iron out, but I am getting there! The piece is 12 mins 52 secs long and has 111 notes played including the pedals - I need to make some pedal meshes when I have some time, right now I need sleep.

    Cheers, Clock.

    PS. Note the node tree execute time of just a smidge over 2ms......

    EDIT:

    Here's just a small section from the MIDI files of all the tempo changes:

    Click image for larger version. 

Name:	Screen Shot 2017-10-25 at 21.04.30.png 
Views:	6 
Size:	78.4 KB 
ID:	502951
    Last edited by clockmender; 25-Oct-17 at 15:05.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  18. #18
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    You can tell I am happy with life and making good progress, because I have gone silly again!

    Here is my Erkinator (Cross between an "Erk" and a Terminator - long story). He has featured in previous projects, generally up to no good, or playing guitar, playing drums, or being rude, etc. He is watching the organ play Thomas Arne's Gavotte, well one of them. I have used my new MIDI nodes to achieve this.

    Click image for larger version. 

Name:	erki-organ.png 
Views:	6 
Size:	650.3 KB 
ID:	503014

    Anyway, I am going to "cook" the animation and post it online over the next few days. The piece is only 2 mins long and I played it into Reason - I now have version 10, sooooo good. My Mk1 MIDI Sync system created the necessary controls to drive the three manuals ("Swell", "Great" and "Choir") and the pedals. There are 75 different notes played across the four channels and it can played with just two hands and two feet.

    Here are some of the Nodes:

    Click image for larger version. 

Name:	Screen Shot 2017-10-26 at 14.03.56.png 
Views:	11 
Size:	120.7 KB 
ID:	503016

    My next challenge is to get Erkinator to play it - this may take some time and considerable brain power.

    Cheers, Clock.

    Edit:

    If you are wondering why there are no "Stops" on the organ - it's because I has to pull all the stops out to get this finished on time.... sorry old English joke!
    Last edited by clockmender; 28-Oct-17 at 12:25. Reason: Bad English.....
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  19. #19
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    Latest Test Video - Erki is enjoying some "leaves" whilst watching the organ play itself, next major goal is to get him playing this tune:



    This is a good test for my MIDI Nodes - very fast play rate and multiple channels. Thanks to Jacques for the Sound Bake Node that was used to make the columns on top of the organ. Oh, and thanks to Thomas Arne for the tune.

    Cheers, Clock.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



  20. #20
    Member clockmender's Avatar
    Join Date
    Nov 2014
    Location
    East Leicestershire, UK
    Posts
    2,549
    More Progress,

    So now I have the alternative method working correctly - this method uses a new Blender CollectionProperty Class to store F-Curves in a list that the AN node tree can use to animate objects.

    Click image for larger version. 

Name:	Screen Shot 2017-10-28 at 18.11.41.png 
Views:	8 
Size:	381.9 KB 
ID:	503344

    This test piece uses the Curves to rotate the keys and also to raise the string dampers. I have introduced a new factor - the "Spacing" that my node uses to space out consecutive notes so they look like they were played twice - yep if I had two consecutive notes of the same value (C4 for example) in the MIDI file, the key was not released in the animation before, so now I can separate them by a number of frames.

    I am now happy both systems are working, just need to do some more testing ad bug checking.

    I hav also realised that I can use this system to animate things, not necessarily to music. So I could create a MIDI file from Reason where I press a key every time I want an object to tilt, for example, then use my nodes to animate it rather than keyframe what could be hundreds of sequences - I just "play" the sequence on my keyboard and my nodes do all the animation in one quick process..... Now there is a thought for complex animation sequences.

    Cheers, Clock.

    EDIT:

    Here's the new node:

    Click image for larger version. 

Name:	Screen Shot 2017-10-28 at 18.31.18.png 
Views:	8 
Size:	109.2 KB 
ID:	503346

    And the animation nodes:

    Click image for larger version. 

Name:	Screen Shot 2017-10-28 at 18.36.22.png 
Views:	7 
Size:	206.2 KB 
ID:	503347
    Last edited by clockmender; 28-Oct-17 at 12:37.
    K.I.S.S - a good maxim I think, and Granddads know best!
    "You are completely bonkers." - @Gumboots
    Youtube Channel https://www.youtube.com/channel/UCfO...1sCoG0NqZkgaAg



Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •