Image/Object alignment solve for camera position/perspective

Coding Python Support
1

I need some help setting up an SVD matrix math problem. I have “limited” experience with this. I was able to follow some other examples to implement an iterative closest point alignment between two objects. Now, I want to use a similar workflow to solve for camera orientation. I’m not sure if I need to know the camera sensor/lense/focal length info or not or if that information can come out of the solution.

here are two examples, the 2nd of which is practically worked out for me, however I am stumbling a bit with it.

http://www1.cs.columbia.edu/~atroccol/3DPhoto/3D-2D_registration.html

Here I wrote a simple UI, in anticipation of figuring this junk out.
https://github.com/patmo141/odc_public/blob/master/py_examples/modal_draw_image_editor.py

So let’s assume I have my 6 x pixel coordiantes and my 6x3d vectors… can anyone help me set up the systems of equations and matrices as described in the 2nd link.

I will post my first attempts here tomorrow evening.

2

I am not sure but, maybe, you can find some ideas-insights in this addon (BLAM) .

3

Thank you. I have investigated BLAM and certainly that is very related addon, but it’s quite different in how it extracts the solution for the camera. I have read the code one time, but not enough to fully comprehend. It may be more similar than I realize, but I think because it uses vanishing lines, instead of individual points, the math is quite different.

-P

4

Ok, updated code can be found here. Even though I plan to use this in my dental addon, I have structured this py file so that it can stand alone.

Here is the relevant code that I am trying to solve the matrix system for.


def build_matrix(self):
        
        if len(self.pixel_coords) < 3:
            print('not enough image points')
            return
        elif len(self.points_3d) < 3:
            print('not enough 3d points')
            return
        
        L = min(len(self.points_3d),len(self.pixel_coords))
        
        #center the points?
        #scale so that mean distance to center is sqrt(3) and sqrt(2)
        
        
        mx_rows = []
        
        for i in range(0,L):
            X,Y,Z,W = self.points_3d[i].to_4d()
            x,y,w = self.pixel_coords[i].to_3d()
            
            r0 = np.array([0,0,0,0,-X*x, -Y*w, -Z*w, -W*w, X*y, Y*y, Z*y,W*y])
            r1 = np.array([X*w, Y*w, Z*w, W*w, 0, 0, 0, 0, -X*x, -Y*x, -Z*x, -W*x])
            
            mx_rows.append(r0)
            mx_rows.append(r1)
            
        A = np.vstack(mx_rows)
        print(A)
        print(A.shape)
        
        u, s, vh = np.linalg.svd(A)  #might need to be A.P to solve for P
        
        print(s)
        print(max(s))
        print(u.shape)
        print(vh.shape)
        '''
        New in version 1.8.0.


        The SVD is commonly written as a = U S V.H. The v returned by this function is V.H and u = U.
        If U is a unitary matrix, it means that it satisfies U.H = inv(U).
        The rows of v are the eigenvectors of a.H a. The columns of u are the eigenvectors of a a.H.
        For row i in v and column i in u, the corresponding eigenvalue is s[i]**2.
        If a is a matrix object (as opposed to an ndarray), then so are all the return values
        '''
5

Can anyone who is good with NumPy help me understand what I am doing wrong?
code: https://github.com/patmo141/odc_public/blob/master/image_object_registration.py
blend: https://www.dropbox.com/s/rqk9rrh9a4rw9ez/image_object_registration.blend?dl=0