Recording (keyframing) realtime animation from drivers (xinput)

Support Animation and Rigging
,
1

I have started to do realtime animations using Erindales XinputReader, which I find super easy to use and great. For a while I just recorded my animations realtime as well, with Obs, but I would like to try some more heavy renderings using this technique.

ChatGpt helped me write the script bellow, but it kills the driver in the keyframing process. So I am not sure how to continue on this. Apparently, drivers are calculated after keyframes in Blender, so a problem is that the recorded value from the driver ends up being 0 for each frame, although I am manipulating the driver in realtime while trying to record keyframes. I also tried parenting an object to an empty controlled by xinput (xbox-controller) and just recording the childs position, but I just get the value 0 recorded for every keyframe.

Does anyone have a suggestion what I can try next? Maybe itā€™s possible in geometry nodes?

Hereā€™s the script:

import bpy

def record_driver(scene):
# Access the object by its name
obj = bpy.data.objects[ā€œObjectNameā€]

# Access the shape key using its name
shape_key = obj.data.shape_keys.key_blocks["ShapeKeyName"]

# Evaluate the driver to get the current value
driver_value = shape_key.driver_add("value").driver.evaluate()

# Apply the driver value to the shape key directly
shape_key.value = driver_value

# Insert a keyframe for the shape key value
shape_key.keyframe_insert(data_path="value")

Clear any existing frame change handlers to avoid duplicates

bpy.app.handlers.frame_change_post.clear()

Register the handler to run on frame change

bpy.app.handlers.frame_change_post.append(record_driver)

2

IMO, thatā€™s your first mistake, using chatgptā€¦

because this is invalid

# Evaluate the driver to get the current value
driver_value = shape_key.driver_add("value").driver.evaluate()

The ā€˜shape_key.driver_add(ā€œvalueā€)ā€™ is wrong. Since itā€™s wrong, it appears to reset the driver. And the 2nd part - ā€˜.driver.evaluate()ā€™ is invalid as well. blenderā€™s not even looking at that bit.

I understand what the XinputReader does, but Iā€™ve never worked with it before. Can you use it to control bones in an armature?

Can you record the emptyā€™s position? And it will get key frames?

If so, I would use the empty as a driver for the shapekey.

Care to explain what you are doing it a bit more?

Randy

1 Like
3

xinputReaderExpl
xinputReaderExpl960Ɨ540 69.2 KB

Thanks for your reply revolt_randy, Iā€™ll try to explain a bit better.

Window 4 in the image (my annotations went away, but it is the fourth window from the left) shows the plug-in xinputReader. It is accessible in geometry nodes as seen in window 1 and 2. To make a new driver, I copy one of the inputs from window 4 by right clicking, selecting ā€œCopy as new driverā€, and then pasting it as a driver in the value I want to control. In this example, I am using the LeftThumbY to manipulate the Z-value of the cube.

I could also control bones directly, but tend to use an empty as an IK-solver for animating bones. So one could plug the outputs into most values.

Right now I just try to record the input I am feeding in in realtime in some way, but the ā€œkeyframe this valueā€ option seems to disappear as soon as I create a driver. The conflict as I understand it is that drivers should be calculated after keyframes in Blender.

Recording the emptys position with auto keying doesnt work, and the way I understand it is, it is because the keyframe is recorded first, and then the driver, so it resets to 0, effectively not recording any values at all.

4

Hereā€™s what Iā€™m thinking - you need a way to insert keyframes while blender is playing the current animation.

Copy this code into a text editor and execute it. (in a new default file)

import bpy


def my_handler(scene):
    print("Frame Change", scene.frame_current)
    bpy.data.objects['Cube'].keyframe_insert('location', index=2)
    
    if scene.frame_current == 100:
        bpy.ops.screen.animation_cancel(restore_frame=False)
        
    


bpy.app.handlers.frame_change_pre.append(my_handler)

Whenever the frame changes, blender will insert a keyframe on the cubeā€™s Z location. If you change the frame by the left or right arrow keys, itā€™ll insert a keyframe. If you press space bar, it will run thru the animation, inserting keys, till it hits frame 100, then it will cancel playback.

This is very crude and missing parts. Like there is no way to cancel that bit of code, it will always run.

Anyway, as proof of example, hook your controller up to the cubeā€™s z-loc, run that code and move the cube around. Once done, save the file. Restart blender, and without executing the script, just playback the animation. It should be the animation you just created.

To be useable, you need a way to cancel the app handler. Maybe as simple as

bpy.app.handlers.frame_change_pre.append(my_handler)

Right after the if statement, but I donā€™t know and donā€™t have time to test.

From here then, you can use the cubeā€™s z-loc as an input to your driver.

Randy

PS - Welcome aboard and forgot to ask, do you know python?

1 Like
5

Thatā€™s very neat! Works quite well, thanks.

Thank you! I donā€™t know much python, but I think I can manage to adapt your script for a more complex scene.

The recorded animation didnā€™t play after reopening Blender, I think because the driver is then overriding it, but I just make an action from it in the NLA-tab and copy that to another cube, or I delete the driver on the first cube, and then it works.

6

So I was thinking about this a little moreā€¦

1st, this is wrong:

it wonā€™t work because there are 2 errors there. 1st error is instead of ā€˜appendā€™ it should be ā€˜removeā€™. The 2nd error is I was suggesting putting that code in the app handler and Iā€™m thinking app handlers canā€™t cancel themselves, the code would need to be outside of the handler part.

Anyway, the real reason why Iā€™m posting now is I thought about this a little more and hereā€™s how I would work thisā€¦

Create a .blend file that has bones controlling all the animation. Save the file as ā€˜something_playback.blendā€™. Then add in the drivers on the bones for xinput and the python script, save this file as ā€˜something_record.blendā€™.

Now, to record your animation, youā€™d use the ā€˜something_record.blendā€™ file. Once you have recorded your animation, you save that file. Open the ā€˜something_playback.blendā€™ file and append in the armature action from the other saved file and it should work.

I suggest using an armature because it is one object. Actions (keyframes) are stored on the object. So 1 armature object with 20 bones = 1 action. 20 empties objects (instead of bones), means you would have 20 actions to appendā€¦

Randy

1 Like
7

Hi again Randy,

Itā€™s a good idea to have two files and append the action. It all seems to work very well, but I have ran into a weird issue. As you can see on the image, the bones seem to have been keyframed correctly. However, the appended action doesnā€™t play. Any idea what might be going on here? You can take a look at the project if you like. I am trying the approach on a more complex rig.

planteIssue
planteIssue1871Ɨ968 410 KB

https://drive.google.com/file/d/1VYGGNgParnQPZxB7Z_UeUXfFbxnZ2dN1/view?usp=sharing

8

Can you post up the .blend?
Randy

Blender Artists |

  • |

rEndre
September 11

Hi again Randy,

Itā€™s a good idea to have two files and append the action. It all seems to work very well, but I have ran into a weird issue. As you can see on the image, the bones seem to have been keyframed correctly. However, the appended action doesnā€™t play. Any idea what might be going on here?

9

Yes, it is in the Google drive link.

Endre

10

Yea, sorry I didnā€™t see the file. My last reply I sent thru e-mail, that why itā€™s all wierd. Iā€™ll never use that method againā€¦

Anyway, it appears your xinput drivers are still on the bones in that file. I looked at the IK_R bone and it has a driver on itā€™s y-loc to RightThumbX. I assume removing the drivers and itā€™ll workā€¦

Randy

1 Like
11

You are right! Not sure how I could overlook that. This is great, thank you so much for your help! I have been thinking about how cool it would be to be able to record those keyframes from this method for a long time.

Endre

1 Like