That is some awesome piece of machinery that you have there and I love the tunes, looking now as I have have at some of your works you have a really wonderful imagination.
My English teacher at school once wrote on my report that my essays “Showed an over active imagination” that went down well with my parents - NOT! :mad: I am enjoying this project, although it is certainly challenging to get the animations working properly. I am developing new methods all the time (for me) with this one and can post some of these if anyone is interested. I am currently developing a MKII Erkinator - the current one is a little unwieldy as a drummer. I have simplified the meshes and modified the armature so he is easier to pose in quick time animations.
I have now got the Hihat working properly so when you raise the top symbol, it gives a different sound.
Video of first 1000 frames of Whiter Shade is on the way - the render is currently keeping my Quad-core warm.
I may change the audio later - any suggestions for this? It will involve a new MIDI file and tweaking the Armature to follow the drum animations, but should take far less time in the future.
But no amp or guitar lead - he hasn’t learnt to play in tune yet and it sounds bloody awful…
I am currently working on a technique to get his left hand to move in response to notes played in the MIDI file, not easy, but I am getting a few ideas. I can get his hand to move along the neck, even when the guitar moves, this was not too difficult. I just need it to be in the right place for the note being played…
Getting his right hand to move when a note is played was fairly simple, I just converted all the notes in the MIDI file to be all one note (middle C) - then ran that through the famous 2.49 script and animated an empty - I then made his hand copy the rotations of that empty.
There will be a video once I have the left hand working correctly, which may take some time. He is to play the first guitar solo from “Comfortably Numb” - I have made the audio (I played this on my guitar first and then converted it to MIDI), just need to sort his left hand.
Erkinator is now playing the drums a little better, i.e. not leaving out the symbols! :eek:
I think he has a little devil in him, but we will just have to live with that. :evilgrin:
On the matter of him playing guitar, I have method whereby his hand bone has a Parent to an Empty. The Empty has a Follow Path constraint with a straight line Bezier curve as the path (just two vertices), which is parented to the guitar neck. This will move his hand to the appropriate fret. I now have to animate the offset of the empty so that when a note is played in the MIDI file, the empty moves to the correct location. This is proving difficult - I can animate the notes, but they then need translating to a fret position, this I can do manually, but I need an automatic routine so that, for example, the second E on the top string is on fret 12, the empty moves to that fret. A further requirement would be to know which string the note is on so he can pluck that string and move his left hand finger also to that string. :spin:
I have got as far as making a curve to represent which fret is needed, then reading the position of the vertex, but this is way too complicated and an object moving along this curve does not move in a linear fashion with time - D’Oh. The other thought is to add a custom property to each note played that says which fret and string it is on, or its offset from the guitar nut, then use these values in a driver for the positional empty, but I would not want his hand to return to the nut after each note - it should move to the next note played, so this is proving difficult as well. A simple Python script that could read the Z values of a shape, a “Blocky” curve if you like, that would tell the Empty where to go on its path would help greatly, but this is beyond me just now.
So this shape might represent the left hand position over time:
And I would want to read the vertices’ Z locations at regular frame intervals that I can set so maybe read the first vertex at frame 1 and the subsequent ones every 20 frames. Making this shape from the notes is also something I need to work on. Has anybody got any ideas here?
Extract the guitar solo from my Reason project and export as a MIDI file.
Create empties for each note played in Blender 2.49 - then run the famous script to get animations of these notes. The important thing here is that they move from Z = 0 to Z = 1 when a note is played, then return to Z = 0.
Save 2.49 project in 2.75 and append empties to my project file.
Create a straight line Bezier from the nut to the 24th fret - just two vertices. Add a “hand-position” empty with a Follow Path constraint to the Bezier. The start of the Bezier and its origin is the guitar nut.
Add custom properties to each note empty giving fret number, string number and proportion along the curve, so for example the 12th fret is 0.666 along the curve, which is also half way to the guitar bridge, Bottom E string is “string-1”, this will be used to move his right hand to pluck this string and his left index, or second finger to hold this string over the 12th fret.
Add an empty called “fret-finder” in the image above, add a Child Of constraint to it for each note, with the respective notes as the targets. Add a driver to each influence field of the constraints, which reads the fret proportion value from the appropriate note object as it’s driver variable - this will raise the “fret finder” by the proportion of the fret position, so for the 12th fret the "fret-finder is raised 0.666 units. The 24th fret will have a proportion of 1 and will raise the “fret-finder” one unit.
Add a driver to the Follow Path constraint on the “hand-position” empty offset field with the “fret-finder” as a target in Transform space with its Z location being the scripted expression value.
EDIT: 7) Add a Limit Location constraint to “fret-finder” as the last constraint in the stack to stop it dancing around all over my screen as the various Child Of’s kick in!
Now when each note is played, the appropriate Child Of constraint kicks in and raises the “fret-finder” by the required proportion, which in turn moves the “hand-position” empty along the fret board. His hand is parented to that, thus moving the hand. I can then take the string number value to move his finger to the required string and also use this to make him pluck the right string. Simples!!! - NOT!!!
Cheers, Toymaker
PS. It is worth noting on a guitar that most of the notes can be played in a variety of positions on the various string and frets, I have used the ones I played rather than stylise it to play only the lower notes up to top E string then run up just that string to the top note. If I make empties for all possible notes on the guitar, I will be able to get him to play any tune. That’s a total of 144 empties but I will not need all of them I am sure…
…and here is a short extract of the solo, showing where his second finger is as he follows the MIDI file. I have also animated his right hand - you can see the plectrum moving to the correct string and plucking it (he said carefully).
I did it in Black and White, at low sample rate, just to see how it is progressing. I am currently doing a full render of the complete solo, it is going to take about 60 hours to cook.
Cheers, Toymaker.
EDIT:
Just spotted a mistake in it - one of the constraints was turned off… D’Oh
Looks fun. Erkinator reminds me of one of the band members in “Captured by robots”. I’m expecting you’ll bake sound to animate those speaker cones? Unlike the EQ with narrow bands, you’ll use a larger range representative of what the speaker can play.
Also on the keyboard being a little off, probably change the easing type for the animation curves to ease-out. I think that will give a faster start to the keydown action. Even if scripted, I wouldn’t be surprised if it’s still doing the default ease-in-ease-out on the animation curves which makes the playing appear soft.
Thanks @pauljs75_ - I will bake the sound to the speaker cones, that’s a good idea and one I had not thought of. I will also animate Erkinator so he moves more like a rock guitarist, although he won’t stand still and just play brilliantly like David Gilmore - I am afraid to say that it is me playing the guitar solo, so it won’t sound as good as the Pink Floyd version.
I have refined the technique so I need far fewer empties and I can easily change the tune now. I have one empty for each fret position, which are driven by the notes through a look-up table that translates notes into string and fret position, His left hand is then placed by these empties. and the right hand is controlled by the same table to get him to play the correct string. I have then animated a further empty called “pluck” which has a very short unscaled curve to get him to just pluck the string at the start of the note. For the “slides” up the keyboard where I have not actually plucked the string, I have added some more keyframes to the note curve that just slide his hand up the neck without him plucking the string. So now to get him to play another tune, I just make the MIDI file, run the 2.49 script and append the new note empties into the 2.75 project, the fret position empties then pick up the fret and string from the lookup table and he can play the new tune. For the right hand controller, I use the 2.49 script to animate all the notes in the MIDI file to the one empty.
I hope all this makes sense! I sometimes get confused myself as to how it works, hence him playing the wrong strings sometimes in the last video, I forgot to activate one of the constraints before I rendered it - D’Oh.
Cheers, Toymaker.
EDIT:
On the keyboard animation, I noticed that the 2.49 script scales the lead-in and lead out of the format f-curve, I have now refined a method to put these back where they belong so the keys get pressed down quickly and release quickly, that “Entertainer” video was early stages of my learning curve and things have improved a lot. I am going to add that piece of music to the “Clockophone” and get the keyboard and maybe some other parts animated properly to the music. I shall also add Erkinator on drums and guitar to the Clockophone project so he plays alongside the main machine. Eee Gad - does that mean there will be two of them? I hope he is not going to reproduce too much…
Makes no sense to me at all whatsoever and yes I am very confused. :spin:
But it is looking very nice and I guess that is all that counts. As long as you know what is going on I am happy. :yes:
In the little vid of the guitar the movements look at bit jerky, is there any way to soften these up, sorry if this is maybe a stupid question that you may already have answered, but as I said earlier I am very confused by what you are doing here with these MIDI files and scripts and what not. (For me a script is something you take to the chemist to get medicine)
This first one shows a section of the solo - these are the F-curves from the note empties, as animated by the 2.49 script. You can see they have very little easing. This was because I played the notes with very fast transitions between one and the next. You can see that the transition from the end of one note to the start of the next is only around 4 frames or 1/6th of a second. If I change the easing, as in the second image, the curves are smoother but the transition time is still only 1/6th of a second and, to be honest not even noticeable in a video. Also the notes are only between 4 and 6 frames long between transitions in this section, so only up to a quarter of a second. Basically I am not sure what I can do to make it smoother.
One issue is that I only used one finger for Erkinator to play the notes, whereas I used all four of mine. To get the animation really accurate I would have to assign each note to a finger, then get the left hand to play each finger properly, that would lead to less movement of the hand and a much more realistic video. I don’t know how practical this is yet as I haven’t tried it, I suspect not because I would not always play one note with the same finger in this solo, or even on the same string. Middle C can be played on String 2 fret 3 or String 1 fret 8 - you see the problem. Maybe I should see if it might work… :eyebrowlift:
Herewith the latest video:
The last notes are played on open A string, so I left his hand on the nut - the guitar nut not one of his… I will do a better resolution video once I am happy with the animations and have animated Erkinator a little.
Cheers, Toymaker
And yes I am confused at times as to how this thing works…:spin:
But first I have a question, if someone knows the answer, I should be very grateful, he said in his best Queen’s English…
Is it possible to take an F-curve from one object and assign it to another at the same point in time?
The reason for this is that I have invested a lot of information in the note empties that I have now for this tune, so Erki plays the correct string and fret. When I want a new tune, it would make life much easier for the animation, if I simply produced a new set of empties for the new tune and then copy their F- curves to the existing note empties. Can this be done, or should I just clear the keyframes from the existing note empties and add Copy Location or Transform constraints to make them follow the new tune empties?
Thanks in anticipation, Toymaker.
EDIT:
I am mow standing in the corner with the Dunce hat on…
I did not even think of Copy & Paste the keyframes from one curve to the other… D’oh
I have also worked out how to make a Sound-Light unit that flashes lights in time with the music across several frequency bands, like the one I built at University using electronics…
Erki under stage lights - one purple, one green, any observations? I may do the next video in this light, unless the collective wisdom is something like “That look s*** Toymaker!”…
The guitar lead is now animated with the guitar, I have added hooks to the last two vertices with the guitar body as the target, the closest to the guitar has a strength of 1, the other 0.5.
The speaker cones now have the sound baked to them.
He can now play chords - I have a series of empties - one for each chord, which when moved 1 unit in Z, move his left hand & fingers to the correct strings/frets and also strum the strings with his right hand.
Here he is playing C major, he has only just leant to play chords, hence he is looking at the guitar neck: :eyebrowlift:
Erkinator is looking great.