Hello everyone,
I’m working on an advanced camera synchronization project using precise data from a robotic arm. I’m injecting absolute coordinates (X, Y, Z, Rx, Ry, Rz) for every frame into a Blender camera rig using a Python script.
frame_time X Y Z RX RY RZ
1764698292.15 -19.144 -31.328 -335.295 -0.068 0.006 -0.014
1764698292.19 -19.153 -32.012 -335.067 -0.166 0.035 -0.026
1764698292.23 -19.3 -33.737 -334.469 -0.406 0.086 -0.061
1764698292.27 -19.416 -35.129 -333.946 -0.632 0.134 -0.089
1764698292.31 -19.63 -37.252 -333.207 -0.926 0.193 -0.127
1764698292.35 -19.922 -40.807 -331.992 -1.462 0.305 -0.203
1764698292.39 -20.243 -44.569 -330.717 -2.02 0.416 -0.284
1764698292.43 -20.691 -49.544 -329.039 -2.683 0.546 -0.38
1764698292.47 -21.019 -53.571 -327.662 -3.229 0.662 -0.46
1764698292.51 -21.437 -59.109 -325.802 -4.031 0.83 -0.588
1764698292.55 -21.919 -64.79 -323.914 -4.855 0.992 -0.71
1764698292.59 -22.427 -71.069 -321.793 -5.74 1.17 -0.846
1764698292.63 -22.924 -76.894 -319.807 -6.553 1.325 -0.98
1764698292.67 -23.379 -82.678 -317.89 -7.349 1.482 -1.104
etc.
coords in mm, rotation in degree
The robot’s coordinate system reference (its World Origin) is set on the filmed object relative to the lens (the robot’s TCP point).
The Problem: Rotation and Scale Drift
After injecting the coordinates, the virtual camera’s movement and rotation seem generally correct. However, a significant “sliding” or “drifting” effect occurs: the filmed object visibly shifts relative to any added Mesh object within the virtual camera’s viewport.
This issue is particularly noticeable in Perspective mode. The alignment is slightly better when I switch the virtual lens to Orthographic, but even then, the Mesh object still shifts incorrectly relative to the background when the camera moves.
This suggests an issue with:
- Imperfect rotation/axis conversion** (Rotation Drift).
- Incorrect scale or focal length (Intrinsics Calibration Error).
My py script:
import bpy
import csv
import glob
import os
import numpy as np
import logging
import math
# --- SCENE CLEANUP ---
bpy.ops.object.select_all(action='SELECT')
bpy.ops.object.delete()
# Delete all materials and images
for block in bpy.data.materials:
bpy.data.materials.remove(block)
for block in bpy.data.images:
if block.name != "Render Result":
bpy.data.images.remove(block)
# --------------------------------------
zoom = 2
lens = 1.082
sensor_width = 9.76
scale_factor = 1
format = 3/4
# ------------------------------------
csv_path = "/home/DanForester/Containers/COORDS_SYNC/synced_output/"
frames_path = "/home/DanForester/Containers/COORDS_SYNC/TIFF_16bit/"
# load coordinates
def load_csv_from_folder(folder):
files = glob.glob(os.path.join(folder, "*.csv"))
if not files:
raise FileNotFoundError(f"No CSV files found in directory: {folder}")
# take the first found file (or you can sort and select the newest)
csv_file = files[0]
cols = (1, 2, 3, 4, 5, 6) # meaning X, Y, Z, RX, RY, RZ
print(f"Loading: {csv_file}")
return np.genfromtxt(csv_file, delimiter=",", skip_header=1, usecols=cols)
# load frames into a TIFF list
def load_tif_from_folder(folder):
files = sorted(glob.glob(os.path.join(folder, "*.tif")))
if not files:
raise FileNotFoundError(f"No TIF files found in directory: {folder}")
print(f"Loaded {len(files)} TIF files from directory: {folder}")
return files
"""
# Scene scale to mm
bpy.context.scene.unit_settings.scale_length = 0.001
# Scene unit of measure
bpy.context.scene.unit_settings.length_unit = 'MILLIMETERS'
"""
# create array with coordinates
cam_coords_data = load_csv_from_folder(csv_path)
# set animation length
bpy.context.scene.frame_end = len(cam_coords_data)
# create a new empty object
#bpy.ops.object.empty_add()
#empty = bpy.context.active_object
# create camera object
bpy.ops.object.camera_add()
camera = bpy.context.active_object
# enable transparency in render (tiff with alpha)
bpy.context.scene.render.film_transparent = True
# calculate sensor height
sensor_height = sensor_width * format
# set sensor
camera.data.sensor_fit = 'VERTICAL' # Use 'VERTICAL' to enforce aspect ratio.
camera.data.sensor_width = sensor_width
camera.data.sensor_height = sensor_height
# camera rotation offset relative to coordinate system
camera.delta_rotation_euler[0] = -1.5708
camera.delta_rotation_euler[1] = 3.14159
camera.delta_rotation_euler[2] = 1.5708
# set camera as child of empty
#camera.parent = empty
# assign coordinates to frames
for i, row in enumerate(cam_coords_data, start=1):
# position with coordinate system axis conversion X, Y, Z - robot
location = (float(row[2]/1000), float(row[1]/1000), -float(row[0])/1000)
#location = (float(row[2]), float(row[1]), -float(row[0]))
camera.location = location
# save position for frame i
camera.keyframe_insert("location", frame=i)
# rotation (degrees → radians)
# Mapping: X_Blender <--- -R_Y_Robot (row[4])
# Mapping: Y_Blender <--- -R_X_Robot (row[3])
# Mapping: Z_Blender <--- R_Z_Robot (row[5])
rx = math.radians(-float(row[4]))
ry = math.radians(-float(row[3]))
rz = math.radians(float(row[5]))
# rotation relative to the origin of the coordinate system
rotation_euler = (rx, ry, rz)
camera.rotation_euler = rotation_euler
# save rotation for frame i
camera.keyframe_insert("rotation_euler", frame=i)
# remember first frame - needed to set and scale the plane
if i == 1:
start_location = tuple(-v for v in location)
start_rotation = tuple(-r for r in rotation_euler)
# create plane
bpy.ops.mesh.primitive_plane_add(size=2, location=(0,0,0))
plane = bpy.context.active_object
# create empty material named tif_Sequence_Material
mat = bpy.data.materials.new(name="tif_Sequence_Material")
mat.use_nodes = True # enable node system for the material
nodes = mat.node_tree.nodes # create access to nodes
links = mat.node_tree.links # create access to connections between nodes
# fetch frames
tif_files = load_tif_from_folder(frames_path)
# compare number of frames
if len(cam_coords_data) != len(tif_files):
print("Warning: number of frames in CSV and TIFF differs!")
# add texture to material
tex_image = nodes.new("ShaderNodeTexImage") # create a new Image Texture node in the material, used to load images (e.g., tif, JPG, tif) and treat them as a texture.
img = bpy.data.images.load(tif_files[0]) # add the first tif file from the list (tif_files[0]) and assign it to the node
w, h = img.size # get resolution
# ---- set render aspect ratio
bpy.context.scene.render.resolution_x = w
bpy.context.scene.render.resolution_y = h
bpy.context.scene.render.resolution_percentage = 100
# -------------------------------------
img.colorspace_settings.name = 'Non-Color'
img.source = 'SEQUENCE' # set image source mode as sequence of frames
tex_image.image = img # assign image to the node
# set sequence
tex_image.image_user.frame_start = 1
tex_image.image_user.frame_duration = len(tif_files)
tex_image.image_user.use_auto_refresh = True
tex_image.interpolation = 'Closest'
# connect to Principled BSDF
bsdf = nodes.get("Principled BSDF") # get the Principled BSDF node from the material tree.
links.new(tex_image.outputs["Color"], bsdf.inputs["Base Color"]) # create a connection (link) between the Color output of the image texture node (tex_image) and the Base Color input in Principled BSDF.
links.new(tex_image.outputs["Alpha"], bsdf.inputs["Alpha"]) # create a connection between the Alpha output of the texture and the Alpha input in Principled BSDF.
# add the created material "mat" to the list of materials assigned to the plane object
plane.data.materials.append(mat)
# set plane as child of empty
aspect = w / h
plane.scale = (aspect * scale_factor, 1 * scale_factor, 1)
plane.parent = camera
plane.location = (0,0,-0.59)
#plane.location = (0,0,0)
#plane.location = start_location
#plane.rotation_euler = start_rotation
# set to panoramic:
#camera.data.type = 'PERSP'
# set camera sensor width
camera.data.sensor_width = sensor_width
# set focal length
camera.data.lens = lens * zoom
Question is:
Rotation/Scale Drift: Given that Orthographic mode works better than Perspective mode, does this strongly suggest an error in the physical camera parameters (lens, sensor_width/height) or the delta_rotation_euler offsets?
Any insight from users familiar with precise geometric data import and robot calibration would be greatly appreciated!
…
i “only” remembered all the camera calibration pattern (checkerboard) even if already knowing/using some camera parameters to refine this. Or the “T-position” for actors before starting a motion capture session. Even CNS moulding / milling machines use some zero position to calibrate… some times also multiple positions.
Best regards. 