2.9 - About real-world to digital world interfacing

Hi guyz :slight_smile:

This is a wide topic i thought i start ages ago when i knew nothing about all this…

Now it’s time, as i played a bit with it and was as much impressed as frustrated.

This world is the tight and discrete world of reality to vitual3D link. The world in wich you bring your holidays photos that you want to modify in 3D realistic rendering; the world where you bring your awesome medieval-castle video where you want the south tower to collapse. the world where you can transform your video to a 3D mesh around wich you can walk and watch at it for hours without the pressure of other boring visitors…

This world have different names: camera-projection, 3D reconstruction, motion tracking, incrustation, etc… All using various techniques.
Personnaly i like the generic term of photogrammetry. photo because of the real-world image sampling/use and metry for measuring and rationalizing all the possible photos cases.

For me, undeforming a randomly taken photo so that the photographed object retrieves its height/width/depth ratios is of the same process as inserting a 3D hole ( or giant alien donut ) on your car park. It’s interfacing 3D and blender to real life photos.

Recently i had fun with blender camera tracking wich is damn simple and allows many many funny things, and i’ll expose my experience here.
But i have to start with the most basic and recent experience i had: a random photo from wich i need to get height-width aspect-ratios.

So let’s start with photo undeformation ( ahahah a new term :stuck_out_tongue: )
—> start and end:
Here’s what i got at start:

( oookay i see you saying: aaah Carcassonne medieval city… agaiiiin… yes… sorry :stuck_out_tongue: )

I took this pic without taking care of the angle, exposure, stillness, etc…
Just like most of you take photos.
Now i want this to be rotated, non-uniformly stretched, sheared, etc so that the width and height of this pic retrieves their real ratios ( just like if i projected this wall with this strange stained glas window on a plane parallel to the wall…
Here’s the result:

This pic will allow me to easily overdraw by hand a mesh that will reproduce this beautifull stonework, giving someting like this: ( on another window )

Even if this sound quite simple, blender cannot do this automatically.
You can painfully try to get the photo in blender and find the perspective points by hand ( including the camera angles/distances ) or use a nice lil tool called fspy.

I leave you looking at this and will post here if needed ( or asked ) the YT tuts for using this. Here are for my photo my results:

and the file: test.fspy (3.8 MB)

Now import this in blender ( with the fspy importer ) and here comes the fun…

Just make a plane, rotate 90° on… okay… a lil vid will tell more:

Note you have to subdivide the plane for proper mapping. The more the better just because in the original photo, 10cm is not the same number of pixels at the bottom of the window and at the top of the window…

later step is to make a plane the same size, don’t divide it but apply a material with a texture target and bake it with ‘selected to active’ option.
You’ll then get a ( almost ) perfectly undistorted picture :slight_smile:

Happy blending !