Converting external application Matrix to blender matrix

RootDaBoy · July 7, 2022, 1:06am

I’m attempting to convert Roblox CFrame (Matrix) to Blender’s matrix, the purpose of this is to enable camera animation exporting and importing said camera animation into blender, the ultimate goal here is to make motion tracking easier as we will have the camera movement data.

Roblox’s CFrame Matrix looks like this.

Note that the Y axis is up in Roblox, versus Z pointing up as in certain other engines.

Some example Lua Code

 local CF = CFrame.new(
    5,5,5, -- Position
    1,0,0, -- RightVector
    0,1,0, -- UpVector
    0,0,1 -- LookVector
)
print(CF:GetComponents())
--> 5  5  5  1  0  0  0  1  0  0  0  1

ref: https://devforum.roblox.com/t/help-with-cframegetcomponents-function/343952

I found some code online that exports blenders animation as Roblox CFrames, however I had no luck making it work for importing CFrame Data into Blender.

def matrixToCFrameComponents(mat):
# switch z and y pos cause blender uses z-up coordinate system
cframe_components = [
	mat[0][3],  mat[2][3], -mat[1][3],
    mat[0][0], mat[0][2], -mat[0][2],
    mat[2][0], mat[2][2], -mat[2][1],
	-mat[1][0], -mat[1][2], mat[1][1]
]
return cframe_components

Ref: https://github.com/zoebasil/blender-cf-exporter/blob/master/addon.py

This is what a CFrame looks like.

(-19.93419,14.09163,-19.06459,-0.69117,0.31943,-0.64827,-0.00000,0.89701,0.44200,0.72269,0.30550,-0.61999)

tl;dr I want to import a camera animation from Roblox into Blender and animate the camera using Roblox’s CFrame Matrix.

testure · July 7, 2022, 6:08am

this is what all 4x4 matrices look like, unfortunately it doesn’t tell us anything, and the Lua code doesn’t make any sense to me since a 4x4 matrix should have 16 values and yours has 12. I’m guessing they’re actually using some bastardized 4x3 matrix and omitting the bottom row? Either way, I’ll leave that part up to you to figure out. What I can give you is more information about Blender’s matrices that may help you accomplish your goals.

Blender’s 4x4 matrix are composed as follows:

the top-left 3x3 values are the rotation component.
the cartesian direction vectors can be extracted using the first three columns (right, forward, and up- in that order).
the fourth column is the position component
the four values diagonal from top left to bottom right represent scaling, and as such they overlap the rotation components.

so as an example- if you wanted to get the position of a matrix, you could create a vector this way:
location = Vector((matrix[0][3], matrix[1][3], matrix[2][3]))

the up vector (+Z axis in Blender) would be:
up_vec = Vector((matrix[0][2], matrix[1][2], matrix[2][2])).normalized()

remember that scaling is done on the diagonal, so if the matrix is scaled direction vectors need to be normalized.

Once you’re able to figure out how Roblox’s matrices are composed, you should be able to use that information to compose a compatible matrix you can use to multiply against your camera’s transform during export.

RootDaBoy · July 7, 2022, 8:04pm

Hi, thank you so much for taking the time to reply.

From my understanding Roblox does use a 4x4 matrix that consists of positional and a 3x3 rotational components, see below for a detatiled article.

I’ll use the code that you provided to see if I can get something to work and will report back

testure · July 7, 2022, 10:20pm

scanning through that article and it seems like they are expecting users to access the direction vectors via some kind of class attribute/getter, they never actually explain how the 3x3 rotation component is actually composed as far as I can tell

RootDaBoy · July 9, 2022, 1:34am

I’ve went with the route of jua converting the CFrame Data to blender’s matrix within the engine, now I just need to figure out how to load it using a modified version of this script.

import os
import json
import math
import bpy
import mathutils


def importCamera(fileName):

    # Make sure the file exists
    if not os.path.exists(fileName):
        print("The provided file does not exist")
        return

    fin = open(fileName, 'r')
    cameraData = json.load(fin)
    fin.close()

    # Create a new camera
    bpy.ops.object.camera_add()
    
    # Give it an unique name
    newCamera = bpy.context.selected_objects[0]
    newCamera.name = cameraData['cameraName']
    newCamera.data.angle = math.radians(cameraData['hfov'])

    start = cameraData['frameStart']
    end = cameraData['frameEnd']

    # Set the current frame (in case we are on a different frame)
    scene = bpy.context.scene
    scene.frame_start = start
    scene.frame_end = end
    scene.frame_set(start)
    
    xrot = mathutils.Matrix.Rotation(math.radians(90), 4, "X")

    for i in range(int(start), int(end) + 1):
        matrixIn = cameraData['frames']['matrix_world'][str(i)]
        
        frameMatrix = mathutils.Matrix([matrixIn[0:4],
                                               matrixIn[4:8],
                                               matrixIn[8:12],
                                               matrixIn[12:16],
                                               ])
        
        # Apply transpose (matrix layout from row, column to column, row)
        frameMatrixTransposed = mathutils.Matrix.transposed(frameMatrix)
        # Apply matrix mult with the xrot for the Maya/Blender coord conversion
        frameMatrixBlender =  xrot @ frameMatrixTransposed
        # Set camera to frame's position
        newCamera.matrix_world = frameMatrixBlender
        
        # Set keyframes
        newCamera.keyframe_insert("rotation_euler", frame=i)
        newCamera.keyframe_insert("location", frame=i)


if __name__ == '__main__':
    importCamera("C:\RBX\cam.json")

source: https://www.youtube.com/watch?v=s1KfrTP4Cxg

testure · July 9, 2022, 1:55am

if you stored the matrix Blender’s native composition (+Z up, -Y fwd), you can just omit this extra rotation and it should be correct.

RootDaBoy · July 9, 2022, 1:56am

 frameMatrix = mathutils.Matrix([matrixIn[0:4],
                                               matrixIn[4:8],
                                               matrixIn[8:12],
                                               matrixIn[12:16],
                                               ])

Would I have to adjust this at all?

testure · July 9, 2022, 1:59am

if you used Lua to write out a matrix that matches what I described up in my first post, you shouldn’t have to change that at all. since you’re writing camera data it should actually be pretty simple since you don’t have to account for scaling, you just need your up, fwd, and right vectors and the camera’s position. the bottom row of the matrix will always be 0,0,0,1

RootDaBoy · July 9, 2022, 2:04am

This is what the conversion outputs.

function ConvertToBlenderMatrix(data)
    local px, py, pz, xx, yx, zx, xy, yy, zy, xz, yz, zz = data:GetComponents()
    local blendermatrix =
        {
            xx, -zx, yx, px,
            xy, -zy, yy, py,
            xz, -zz, yz, pz,
            0, 0, 0, 1
        }
    
    return blendermatrix
end

local testData = CFrame.new(-9.84016323, 6.96753883, -7.84781408, -0.617346704, 0.350819975, -0.70413661, 1.49011612e-08, 0.895061016, 0.445943713, 0.786691189, 0.275301874, -0.552563012)

print(ConvertToBlenderMatrix(testData))

[1] = -0.6173467040061951,
[2] = 0.7041366100311279,
[3] = 0.3508199751377106,
[4] = -9.840163230895996,
[5] = 1.490116119384766e-08,
[6] = -0.4459437131881714,
[7] = 0.8950610160827637,
[8] = 6.967538833618164,
[9] = 0.7866911888122559,
[10] = 0.5525630116462708,
[11] = 0.2753018736839294,
[12] = -7.847814083099365,
[13] = 0,
[14] = 0,
[15] = 0,
[16] = 1

testure · July 9, 2022, 2:09am

yep, that looks correct to me- write it out to a json and try out your Blender script, if the camera looks like it’s moving down the wrong direction on the forward axis you might need to flip your Z axis back to positive, but I’m assuming roblox is +Z fwd like Unity

edit: the position data might be off, since the world coordinates are based on roblox’s coordinate system- you may need to swap the y and z axes (and possibly negate z). I’d say test out what you have, and if it doesn’t look correct try to adjust your position based on that. It’s hard to think about these rotation differences abstractly without actually testing it out, so I’d just experiment with it a bit

RootDaBoy · July 9, 2022, 2:17am

Hmm, just tried it and I’m guessing I have to flip the Z axis.

Blender:

Roblox Camera:

Edit: Flipped the axis, now its pointing the correct way but there seems to be an odd rotation, I also noticed the camera is only rotating not actually updating its position.

RootDaBoy · July 9, 2022, 5:29am

Got something working!

Only problem is, it seems to be orienting the wrong way my Roblox camera movement is going in circles like this

however blender is going like this

import os
import json
import math
import bpy
import mathutils


def importCamera(fileName):

    # Make sure the file exists
    if not os.path.exists(fileName):
        print("The provided file does not exist")
        return

    fin = open(fileName, 'r')
    cameraData = json.load(fin)
    fin.close()

    # Create a new camera
    bpy.ops.object.camera_add()
    
    # Give it an unique name
    newCamera = bpy.context.selected_objects[0]
    newCamera.name = cameraData['cameraName']
    newCamera.data.angle = math.radians(cameraData['hfov'])

    start = cameraData['frameStart']
    end = cameraData['frameEnd']

    # Set the current frame (in case we are on a different frame)
    scene = bpy.context.scene
    scene.frame_start = start
    scene.frame_end = end
    scene.frame_set(start)
    

    for i in range(int(start), int(end) + 1):
        matrixIn = cameraData['frames']['matrix_world'][str(i)]
        
        frameMatrix = mathutils.Matrix([matrixIn[0:4],
                                               matrixIn[4:8],
                                               matrixIn[8:12],
                                               matrixIn[12:16],
                                               ])
        
        newCamera.matrix_world = frameMatrix
        
        # Set keyframes
        newCamera.keyframe_insert("rotation_euler", frame=i)
        newCamera.keyframe_insert("location", frame=i)


if __name__ == '__main__':
    importCamera("C:\mayatest\main.json")

testure · July 9, 2022, 3:36pm

you should go ahead and post your json output as well, it’s moving into the realm of being easier to massage into working correctly via testing

RootDaBoy · July 10, 2022, 1:27am

This is the JSON output.

RootDaBoy · August 6, 2022, 4:57pm

Hia! I made some progress on this I have some camera data that is read from a JSON file, I’d like to animate the camera based on the Position and Rotation values.

"1": [
 "Position": (-24.129362106323242, -0.5124680399894714, 14.283154487609863),
 "Rotation": (-1.0030092000961304, 3.1415927410125732, -1.4891589879989624)
]

Right Now I have this script to manually set the position but I’m a bit lost on how to get it to make keyframes.

import bpy
import math
from mathutils import Euler

obj = bpy.context.object

data = {
            "Position": (6.474687576293945, 7.203999042510986, 13.555723190307617),
            "Rotation": (-0.5392979383468628, -3.1415927410125732, 2.3038623332977295)
        }
        
#sets ObjectSpace rotation
def set_loc_rotation(obj, value):
    rot = Euler(value, 'ZYX')
    obj.rotation_euler = (obj.rotation_euler.to_matrix() @ rot.to_matrix()).to_euler(obj.rotation_mode)
    return

#parses & applies location + rotation from CFrameToBlenderData
obj.location = (data["Position"][0], data["Position"][1], data["Position"][2])
obj.rotation_euler = (data["Rotation"][0], data["Rotation"][1], data["Rotation"][2])

#fixes y rotation inversion
set_loc_rotation(obj, (0, math.pi, 0))