AI software that makes face movements and keyframes?

Engie · March 2, 2025, 1:35pm

hey

Does anybody know is there AI software that can create keyframes for facial expression without video where to look facemovement? This would be very handy tool since face animation is so hard if there were AI software that jut create keyframes for facial movement like from text. You jsut write what the face animation should so this would include speech and emotion. With this you could then just export keyframes onto your own animation.

Perhaps maybe some sort of face measurement for the face you are using the keyframes would be good too so they would match the face when you import the keyframes from the AI software.

anybody know something like this does it exists already?

joseph · March 2, 2025, 1:52pm

Nvidia Audio2Face

Engie · March 2, 2025, 2:33pm

hoyl moly
Seems it can create facial expression just from audio. This is nice, but then I need AI software to create emotive audio hehe

https://docs.omniverse.nvidia.com/audio2face/latest/overview.html

joseph · March 2, 2025, 2:47pm

We use A2F at work, it works great but we don’t use it exclusively, we always do a manual tweaks pass over the top.

A2F gives you JSON files- to use them, you’ll need this addon:

github.com

josephclaytonhansen/blender-addons/blob/main/audio-2-face-weights-import

bl_info = {
    "name": "A2F Import Weights",
    "blender": (3, 6, 0),
    "version": (1, 3, 0)
}

import bpy
from bpy.props import StringProperty, IntProperty
from bpy.types import Panel, Operator, PropertyGroup
import json

class createAnimation(Operator):
    bl_label = "Create Animation"
    bl_idname = 'object.blendshape_animation_operator'
    
    filepath: StringProperty(name="JSON file", description="JSON blendshape file from Audio2Face", subtype="FILE_PATH")
    multiplier: IntProperty(name="Multiplier", description="Multiplier for blendshape strength", default=1, min=1)
    
    def execute(self, context):
        try:

This file has been truncated. show original

Engie · March 2, 2025, 5:55pm

Requirements

In order to use Audio2Face, you must have:

Win 64 1909 or newer
Omniverse Nucleus
Ubuntu Linux Version 20.04 or newer

I use windows 10 64bit so does that suffice? is Nucleus the omniverse software that runs this 2 face thingie? and does it also requires ubuntu full OS too?`

Why can’t it be just download this one software and voela thing