Script to Mute and Unmute Audio based on a input file

toxsickcity · May 2, 2024, 9:20am

Hello fellow Blenderers,

I hope someone may help me to get a task done, I am stuck having to manually animate mute on and off audio track based on timings of a file,
Basically have blender on 1 monitor and the data file on 2nd monitor and just animate the toggle of mute for the audio of the track.

The blender looks like this when I am doing the work manually,

The Dope Sheet has keyframed audio volume of 0 and 1, the keyframes that are side by side are to stop the volume ramping up and makes it as close to a square pulse as possible.

essentially I have a script file which looks like this,

I have a media clip which is matched to the script which in this case goes for 3 seconds (3000ms) and is 30fps if that matters,

I need to have the audio in the sequencer for the media clip toggle the audio on and off.
“at” repesents to mute and unmute based timing in miliseconds.
“pos” represents Mute on or off, so below 50 will be unmuted above 50 muted.

Script.txt (775 Bytes)

This is how the data is always presented, some goes for longer time such as 60 seconds.

{
    "inverted": false,
    "fps": 30.0,
    "toggle": [
        {
            "pos": 80,
            "at": 100
        },
        {
            "pos": 0,
            "at": 333
        },
        {
            "pos": 61,
            "at": 500
        },
        {
            "pos": 0,
            "at": 833
        },
        {
            "pos": 81,
            "at": 1133
        },
        {
            "pos": 15,
            "at": 1300
        },
        {
            "pos": 79,
            "at": 1800
        },
        {
            "pos": 17,
            "at": 2300
        },
        {
            "pos": 96,
            "at": 2833
        },
        {
            "pos": 40,
            "at": 3000
        },
    ],
}

I know nothing of scripting and wouldnt know where to start, let alone even know if this is possible.

It would make my life so much easier to automate the process.

Thanks
Shaun.

Cornerback24 · May 3, 2024, 3:46am

Took a stab at writing a script because it sounded like a fairly simple problem. Discovered that the Sequence animation data is not actually on the Sequence but directly on the Scene which was interesting, but I did make something so here you go. It should add a button to the Tool panel in the sequencer.
By the way you can set keyframe interpolation to Constant and it will hold the value until the next keyframe.

import json
import bpy
from bpy.props import EnumProperty


# returns a python dictionary representation of the data in the audio muting data file
def audio_data_from_file(filename):
    with open(filename, 'r') as file:
        # ignore any lines that may be just whitespace
        lines = [line for line in file.readlines() if line.strip()]
    # remove the trailing commas to make valid json
    lines[-2] = "]"
    lines[-3] = "}"
    # join all the lines back together and make a dictionary out of the data
    json_string = ''.join(lines)
    return json.loads(json_string)


# converts ms to frames using the scene's frames per second
def ms_to_frames(context, ms):
    return int(ms * .001 * context.scene.render.fps)


# code for the "Add muting keyframes" button
class AddMutingKeyframesOperator(bpy.types.Operator):
    bl_idname = "muting_utli.add_muting_keyframes_operator"
    bl_label = "Add muting keyframes"
    bl_description = "Add muting keyframes"
    bl_options = {"REGISTER", "UNDO"}
    filepath: bpy.props.StringProperty(subtype="FILE_PATH")

    def execute(self, context):
        # read in file with audio muting data to a python dictionary
        audio_data = audio_data_from_file(self.filepath)
        # The animation data is not on the Sequence object.
        # It is directly on the scene with a data path referencing the sequence
        data_path = f'sequence_editor.sequences_all["{context.scene.audio_to_add_muting_keyframes_to}"].volume'
        # just calling keyframe_insert to ensure there is an fcurve, so we can use keyframe_points.insert which
        # returns the actual keyframe
        context.scene.keyframe_insert(data_path, frame=ms_to_frames(context, audio_data["toggle"][0]["at"]))
        # get the fcurve to add keyframes to
        fcurve = context.scene.animation_data.action.fcurves.find(data_path)
        # loop through the audio muting data that can from the file and add keyframes for each toggle
        for keyframe_audio_data in audio_data["toggle"]:
            frame_number = ms_to_frames(context, keyframe_audio_data["at"])
            keyframe_value = 0 if keyframe_audio_data["pos"] < 50 else 1
            # add the keyframe, replacing any existing keyframe at the frame
            keyframe = fcurve.keyframe_points.insert(frame_number, keyframe_value, options={'REPLACE'})
            # hold keyframe value until the next keyframe
            keyframe.interpolation = "CONSTANT"

        return {'FINISHED'}

    # noinspection PyUnusedLocal
    def invoke(self, context, event):
        context.window_manager.fileselect_add(self)
        return {'RUNNING_MODAL'}


# code for the panel that displays the button and sequence selection
class AddMutingKeyframesPanel(bpy.types.Panel):
    # place in Sequencer Tools panel
    bl_space_type = "SEQUENCE_EDITOR"
    bl_region_type = "UI"
    bl_category = "Tool"
    bl_label = "Add muting keyframes"
    bl_idname = "ANIMATION_PT_add_muting_keyframes_panel"

    def draw(self, context):
        self.layout.prop(context.scene, "audio_to_add_muting_keyframes_to")
        self.layout.operator(AddMutingKeyframesOperator.bl_idname)


sound_sequences_names = []


# return a list of sound sequence names that is used for the sequence select drop-down
def sound_sequence_names(self, context):
    global sound_sequences_names
    names = [sequence.name for sequence in context.sequences if isinstance(sequence, bpy.types.SoundSequence)]
    # api says you need to keep a reference to this around to avoid problems for dynamic enums
    sound_sequences_names = [(name, name, name) for name in names]
    return sound_sequences_names


classes = [AddMutingKeyframesPanel, AddMutingKeyframesOperator]


def register():
    # ideally this would be a PointerProperty, but Sequence is not an ID type so that doesn't work
    # also Sequence animation data is directly on the Scene, not the Sequence
    bpy.types.Scene.audio_to_add_muting_keyframes_to = EnumProperty(items=sound_sequence_names,
                                                                    name="Sound sequence")
    for clazz in classes:
        bpy.utils.register_class(clazz)


def unregister():
    for clazz in reversed(classes):
        bpy.utils.unregister_class(clazz)


if __name__ == "__main__":
    register()

toxsickcity · May 6, 2024, 9:56am

Hello Cornerback24,

Thank you so much for your script,
I got it to add the button in blender as you mentioned.
It seems to map mute on a few keyframes like 2-3 of them and ends.

as seen below, I ran script.txt as what was attached and expect the first 3 mute/unmute events at 100ms, 333ms and 500ms (so at frames 3, 10 and 15)

Any ideas what might be going wrong?
I am running Blender 3.6.11. do you think it might be a mismatch version issue?

Thanks again for having a go and helping me

Cornerback24 · May 6, 2024, 1:41pm

Hm I tried it in 3.6.11 and I’m not seeing that. Some things to check:

The script uses the frame rate of your scene (not the “fps” from the file). Is your scene’s frame rate set to 30 fps?
Is there anything that looks like an error in the system console when you try adding the keyframes? (System console is the black window that opens alongside blender. Window > Toggle System Console if you don’t have it.)
Check that it is parsing the file right: add print(json_string) after the line that has “json_string = ‘’.join(lines)”, and add print(audio_data) after the line that has “audio_data = audio_data_from_file(self.filepath)”. What does it print out in the system console when trying to add keyframes after adding those print statements?

toxsickcity · May 7, 2024, 2:40am

Hey Bud,

Thanks for the reply,
I have made the changes to track the progress in the Console,

I have pasted the results below,

EDIT: Would it be worth uploading a blend file with the script in the script tab, the blender file which worked on your system?

cheers

{
    "inverted": true,
    "fps": 30.0,
    "toggle": [
        {
            "pos": 80,
            "at": 100
        },
        {
            "pos": 0,
            "at": 333
        },
        {
            "pos": 61,
            "at": 500
        },
        {
            "pos": 0,
            "at": 833
        },
        {
            "pos": 81,
            "at": 1133
        },
        {
            "pos": 15,
            "at": 1300
        },
        {
            "pos": 79,
            "at": 1800
        },
        {
            "pos": 17,
            "at": 2300
        },
        {
            "pos": 96,
            "at": 2833
        },
        {
            "pos": 40,
            "at": 3000
}]}
{'inverted': True, 'fps': 30.0, 'toggle': [{'pos': 80, 'at': 100}, {'pos': 0, 'at': 333}, {'pos': 61, 'at': 500}, {'pos': 0, 'at': 833}, {'pos': 81, 'at': 1133}, {'pos': 15, 'at': 1300}, {'pos': 79, 'at': 1800}, {'pos': 17, 'at': 2300}, {'pos': 96, 'at': 2833}, {'pos': 40, 'at': 3000}]}

Cornerback24 · May 7, 2024, 3:53am

Sure, attached a blend file that worked on my system.
The attached blend file also has a “Script with print statements” that prints out more detail to the console about each keyframe being added.
The output you posted looked like the file parsed fine so something must be going on after the file gets parsed.
testingkeyframegeneration.blend (944.2 KB)

toxsickcity · May 7, 2024, 11:07am

Hey CornerBack24

Thanks again for your messages and assistance.

I am experiencing the weirdest thing, although I get it to work sometimes with your file,
It appears that if I clear items of the sequence editor then add my video file into a empty sequence editor, then add my script file, I am getting the same issues as before.

However. If I leave your items in the sequence editor then add my new video above (on top) of yours, I can map audio mute to it using different script files (IE:I it working fine)
In the same session of using blender.
Even If when I remove all items from sequence editor and add a new clip to an empty sequence editor, it works fine still, it’s only when I empty the editor right after opening your file then adding new stuff it fails.

when it fails I get this in the console.

Read blend: "C:\Downloads\testingkeyframegeneration.blend"
Warning: region type 14 missing in space type "View3D" (id: 1) - removing region
Warning: region type 15 missing in space type "View3D" (id: 1) - removing region
Warning: region type 14 missing in space type "View3D" (id: 1) - removing region
Warning: region type 15 missing in space type "View3D" (id: 1) - removing region
Warning: region type 14 missing in space type "View3D" (id: 1) - removing region
Warning: region type 15 missing in space type "View3D" (id: 1) - removing region
Warning: region type 14 missing in space type "View3D" (id: 1) - removing region
Warning: region type 15 missing in space type "View3D" (id: 1) - removing region
Warning: region type 14 missing in space type "View3D" (id: 1) - removing region
Warning: region type 15 missing in space type "View3D" (id: 1) - removing region
Warning: region type 14 missing in space type "View3D" (id: 1) - removing region
Warning: region type 15 missing in space type "View3D" (id: 1) - removing region
Warning: region type 14 missing in space type "View3D" (id: 1) - removing region
Warning: region type 15 missing in space type "View3D" (id: 1) - removing region
Warning: region type 14 missing in space type "View3D" (id: 1) - removing region
Warning: region type 15 missing in space type "View3D" (id: 1) - removing region
Warning: region type 14 missing in space type "View3D" (id: 1) - removing region
Warning: region type 15 missing in space type "View3D" (id: 1) - removing region
Warning: region type 14 missing in space type "View3D" (id: 1) - removing region
Warning: region type 15 missing in space type "View3D" (id: 1) - removing region
Warning: File written by newer Blender binary (401.23), expect loss of data!

I am not overly concerned about version error as like I said it’s sort of working if I massage it right! haha

Let me know your thoughts.

Regards,
Shaun

Cornerback24 · May 7, 2024, 2:09pm

That is interesting. I don’t think those region warnings are related either.
I am noticing that when I open up the file and right after empty the sequence editor and then add new mp4 it changes the scene’s frame rate. Maybe that has something to do with it…

If so maybe try checking the scene’s frame rate before adding the keyframes, or in the script replace context.scene.render.fps with 30.

toxsickcity · May 9, 2024, 9:26am

Hello Cornerback24.

Turns out the issue was totally related to FPS,
If I simply select a different FPS and then back to 30, it works fine,
The script is flawless,
Wanted to thank you so much for your help and for the script,

Thanks again,
Shaun

EDIT:
P.S.,
Is there a way to enable a function that reverses the Mute and Unmute based on the script line, of true or false.
“inverted”: false,

Cornerback24 · May 9, 2024, 8:40pm

Yeah that’s easy enough to do, just needs a check on “inverted”:
Replace

            keyframe_value = 0 if keyframe_audio_data["pos"] < 50 else 1

with

            if audio_data["inverted"]:
                keyframe_value = 0 if keyframe_audio_data["pos"] > 50 else 1
            else:
                keyframe_value = 0 if keyframe_audio_data["pos"] < 50 else 1