GSoC 2016 - Multiview Reconstruction Almost Done

http://download.blender.org/ftp/incoming/Blender-multiview-mac-new.zipHi all,

GSoC is coming to an end. My project, Multi-view Reconstruction for Blender, is almost finished and I would like to invite users to try it. First of all, if you don’t know what I am doing, you can have a look at of this blog post:

I’ve also made a video to demonstrate the workflow:

I will post the link to my work product later. Feel free to leave comments here. :slight_smile:

Update (August 27, 2016):
You can now download the mac test build at http://download.blender.org/ftp/incoming/Blender-multiview-mac-new.zip. The color of linked tracks are not shown correctly for this build. But you can follow the workflow and try it. I will upload a stable one in a few days to graphicall.org.

Update (August 31, 2016):

I update a linux build to graphicall.org. Check it out here: http://graphicall.org/1198

However, since I don’t have a windows machine. I would appreciate it if someone can build my branch on a windows machine. The branch name is “soc-2016-multiview”.

To me its unclear followed both links, what is it suposed to do ?.
Please describe it in layman terms 5 lines or so. (if possible)

And a download link to a blender build with what you made in it, might be handy if you like people to test it,

@tianwei37, thanks!

@Razorblade. Seems Camera tracking support for multi cameras, like those used for 3D film.
https://wiki.blender.org/index.php/User:Tianwei37/GSoC_2016_Proposal

Edit:
Now I’m not sure if this is for 3D films, or to use a secondary camera as a helper to get a better result in the tracking.

It makes always happy to see a Gsoc with a good doc and a will to finish the job to make it master ready :slight_smile:
Regarding feedback, the only part that may be to time consuming is linking the tracked points manually. So if you have time after you got the work in master, I would definitely try to get some algorithm from photogrammetry programs to recognize features from different views. I use photoscan but https://github.com/openMVG/openMVG is also pretty good and open source.
If you have some links with usable footages to test your work, I think more user could help.

great job tianwei37, congrats!

would like to know a little more about how this works: can I take a second shot with the same camera from a different angle? or could it be just a picture? would it be better to mimic the movement of main camera or just have a wider shot? seen a couple of video around, is this similar to a survey shot? does it need the two clips to be equal in length and resolution?

and yes, you can get a lot of info from just 2 pictures with photogrammetry

It seems that you need two videos, not pictures. The software needs movement in order to calculate the distance/angle between the different points that you are tracking and probably both need to be in perfect sync. Maybe the author can detail a little more the workflow.

@YAFU Thanks, the new feature is to get a better result in the tracking, because a single clip would suffer from camera jittering or rolling shutter. :slight_smile:

@bliblubli, thanks for the suggestion. I am well aware of openMVG. openMVG and blender sort of share the core library of multiview functions, which is libmv. Actually automatic feature detection is a goal we are chasing for. I also think that an automatic tracking plus a human correction is the correct direction. But manual labeling is current status when I first knew blender. I have great interests in implementing this new feature and will talk with Sergey about this. But it is also a big project and may take a great amount of efforts to achieve this.

@liero Thanks. Yes, you need to take a second shot capturing the same scene from a different view.
Of course you can take the shots with the same camera, as long as they share some overlaps of the scene you are capturing.
I think it can be a picture, but it is better to be a video clip.
It is better to have a wider baseline angle from the main clip.
Mimicking the movement of the main clip is not a good option, since wider baseline angle will increase the accuracy of the solver.
I don’t know what you mean by survey shot but I guess it is it. The two clips don’t have to be equal in length and resolution.

@Birkov Thanks. Yes, we indeed need two video clips here. For more details, you can check out the user doc on my blog post: http://hlzz.github.io/blender3/

@tianwei37, thanks for the answers.
I bump this thread for others can see answers in Today’s Posts, since the thread is not updated for being you a new user.

FYI, you can now download the mac test build at http://download.blender.org/ftp/incoming/Blender-multiview-mac-new.zip. The color of linked tracks are not shown correctly for this build. But you can follow the workflow and try it. I will upload a stable one in a few days to graphicall.org.

May I ask, why a video clip would be better? Because of averaged 2d feature placement errors?

Is it possible to solve a still camera with help of other reference still images?

Good to know you have interest in this. It is indeed a big project. Nobody expect you to do a lot of efforts for nothing. I guess it only make sens if you have fun or if it opens some good jobs doors or both.

I update a linux build to graphicall.org. Check it out here: http://graphicall.org/1198

However, since I don’t have a windows machine. I would appreciate it if someone can build my branch on a windows machine. The branch name is “soc-2016-multiview”. Thanks.

Hi Tianwei37, can you upload the mac build again? The Ftp link in not valid anymore. Thank you.