Camera Calibration using Perspective Views of Rectangles

add-ons

(marcor) #1

Hello

I made a blender add-on for matching the camera position, rotation and focal length to the view in a photograph. All that is required is having a perspective view of a rectangle in the image.

You can find the add-on on https://github.com/mrossini-ethz/camera-calibration-pvr along with instructions about how to install and use it. Here is a short version of the usage for users that are experienced with blender:

  • Use the photograph as a background image in the 3D viewport.
  • Create a new plane mesh.
  • In top view and in edit mode, move the vertices of the plane to the corners of the rectangle in the photograph.
  • Leave edit mode and press the Camera Calibration button in the tool shelf (in the Misc tab).

The image below uses this method to insert 3D shapes into a photgraph. To match the perspective, the grid lines on the paper are used to define a rectangle. In case it is not obvious: The dragon as well as the array of cubes have been added using blender. Credit to the dragon model goes to Stanford Computer Graphics Laboratory (https://graphics.stanford.edu/data/3Dscanrep/).


I am aware that there is another add-on that does practically the same. However, I have never gotten that one to work for me. Consider this as an alternative, especially if you had similar experiences.

The current version is still somewhat experimental. I welcome any suggestions that may simplify the usage or add new features. This thread is not only intended to spread the word, but also to receive feedback to improve the add-on.


Blam addon issue
Blam issue!
Non-orthogonal reference images?
(Spirou4D) #2

Hi Marcor, Thanks you so much for sharing this job!
I am trying…
At soon.
Spirou4D

EDIT: But after Leave edit mode and press the Camera Calibration button in the tool shelf (in the Misc tab).
My camera move at a new position and see me the view but what can I make after that? I see nothing?:eek:

Here is my scene: http://storage7.static.itmages.com/i/17/0119/h_1484833905_3021997_461d9c6a16.jpeg


(marcor) #3

Hi Spirou4D

Thank you for trying the add-on! From the screenshot that you posted, it appears that you are in top view. You said that the camera was moved and the view changed after the button press, correct? The view should have changed into camera mode. In addition, a new mesh (currently named CalRect) should have been created in the x-y-plane. In the documentation, it says that the background image should be added in ‘Top’ mode (obsolete). Therefore, the image is not visible in camera mode. If you change to background image mode to ‘All Views’ instead, you should hopefully see the CalRect being aligned to the background image while flat on the x-y-plane. Does this help you?

I will update the documentation to be more clear on this point. If you found a bug - even better!

marcor


(Spirou4D) #4

Thks for your reply! I obtain this:
http://storage1.static.itmages.com/i/17/0119/h_1484865386_4524977_7ec988127f.jpeg

I have well the “CalRec” object but what I make with it, please?

EDIT: Ha yes I understood! But to view the backdrop image i must choose Front in panel.
And then, Hide the first plan i have draw.
Then I can add a cube and scale it to the background image…

Very good addon! Congrats Marcor!
From where in Italia do you live, please. I know very well Italia, Venezia, Treviso, so Lombardia in general!


(marcor) #5

Does this mean that it works for you now?

Please note that I have made several improvements to the add-on in the meantime.


(Spirou4D) #6

YES very good, no problem, all is OK! Very usefull!

Ok I re-download the new one…Thks and Congrats.
What is the change, please? important or details?


(marcor) #7

Changes and improvements made so far:

  • Fixed a bug that occured under certain conditions leading to an invalid calibration
  • The size of the reconstructed rectangle can now be specified as an option
  • Fixed a bug that in some cases made the update of the camera focal length fail
  • An error is now reported when there is no active camera
  • The name of the reconstructed rectangle is now the name of the input rectangle with a “_Cal” suffix
  • An option was added to allow the reconstructed rectangle to be perpendicular to the x-y-plane
  • Improved the documentation

Please feel free to report any problems with the add-on. Feature requests are also welcome. You can do both here or at


(urkokul) #8

Thanks for good job


(Wig42) #9

Thanks a lot marcor for this. Like you I could never get BLAM to work. This will save me having to use sketchup foe camera matching. Great job.

Cheers Wig


(Safetyman) #10

This is very useful for me… I can’t wait to try it out. Thanks!


(Dito) #11

Hi marcor, looks great.
But I do not understand this
To reposition, rotate and/or scale the generated rectangle, be sure to reposition, rotate and/or scale the camera along with it.

If I reposition both, the Plane and the Camera, nothing changed of course.
Or have I misunderstood it?

I’m trying this image.

But it doesn’t work.
Or I have chosen an “impossible” picture?


(marcor) #12

Hi Dito

The rectangle that is reconstructed in 3D will lie flat on the x-y-plane. If you try to reconstruct the wall of a building for example (which is vertical, normally) you would then have to rotate the rectangle around x or y to make it vertical in blender. You would then have to rotate the camera along with it, otherwise it would not match the rectangle anymore. The view through the camera will of course stay the same.

(Actually, there is now an option to have the rectangle in vertical orientation from the beginning. But you may still want to relocate/rotate/scale the generated rectangle.)

Now about your reference image. You may notice that the vertical lines (in 3D) are actually vertical in the image (in 2D) as well. This means that the camera is either pointed straight at the horizon or that the camera sensor/lens is shifted. Both will reduce perspective distortion. The add-on that I wrote (and others maybe as well) rely on this distortion to calculate the camera position. This might be the reason why the add-on will not work. Another reason may be that the image was cropped (which is essentially the same as a lens/sensor shift) and in that case the optical centre is not in the middle of the image. This will make most algorithms fail as well. Scaling of the image in x- or y-direction only is also problematic, but may not be applicable in the case of your image. BTW, all this is now pointed out in the readme on https://github.com/mrossini-ethz/camera-calibration-pvr.

So, yes, please use another image …

(Actually, I think it may be possible to reconstruct the camera position/rotation from such an image, knowing that there is shifting/cropping, but you would have to use a different approach. I will think about this.)

Does this help?


(Dito) #13

Hi marcor,

that is really a detailed and understandable answer.

Thank you!


(Safetyman) #14

My first go with it produced great results! I can’t believe how easy it worked. Thanks Marcor! Now I just have to git gud at realworld lighting. :frowning:


(marcor) #15

Thanks everyone for the feedback!

I hereby release version 0.1! You can find it here:

https://github.com/mrossini-ethz/camera-calibration-pvr/releases

Here’s more good news:
Through feedback, especially from Dito (see above) and callimero (GitHub), I was motivated to find a way to calibrate the camera for an image where lens-shift is applied! This means that many architecture-like photographs can now be used for calibration. The following screenshot shows how to use the new method:


There are a two restrictions:

  • Two edges of the rectangle must be parallel (as seen in the image).
  • An additional ‘dangling’ vertex must be connected to one of the rectangle corners to indicate the change in direction at the edge (the edge must be 90 degrees in 3D).

Please see the documentation for details:

Now, in addition to camera focal length, position and rotation, one more camera parameter is reconstructed: Vertical shift. It works very well for me, however I have not had the opportunity to test it thoroughly. Please feel free to give me any feedback about it.

I think that this form of camera calibration represents a novelty for blender! I will release it as 0.2 when together we have elimitated the bugs!

@Dito: Try your image again (the one you gave up upon). You will be pleased.


(Dito) #16

Hi marcor,

absolutely great!!!

Looking forward to the 0.2.



(rombout) #17

Came here through the 'other" addon, this looks very promising!!! great work


(rombout) #18

Hmmmm… tried it and got this result. Camera is rotated on X axis somehow

Attachments



(marcor) #19

Okay, that does not look good. However, I have not enough information to fix the problem. Could you open an issue on

and post some details there? Especially a screenshot of the scene in Top View or the .blend file before pressing the “Shifted Perspective” button?

Thanks for your feedback!


(rombout) #20

Yes will do, im curious how that other guy fixed it actually. Perhaps i did somethin wrong, im on osx using 2.78a