I’m a filmmaker, not a 3D artist, and would like to know if I can use Blender’s camera tracker to combine two shots that I’ve filmed into one longer one? I have attempted this with no success. The two camera’s sizes were all out of whack in the 3d viewer.
Shot 1 ends around where shot 2 begins. I’d like to track both, match-up similar tracking points in the 3D viewer, and figure out if there’s anywhere where the two cameras are in the same position. At that spot, end shot 1 and begin shot 2 for a (hopefully) seamless transition–a fake long take.
I realize the odds of the two cameras positions and timing matching up are pretty slim. I’ll probably have to create a 3rd camera to somehow merge the two. But, right now I’m just trying to figure out if this is even possible with Blender.
Based on what you are telling me I will assume the camera’s were not mechanically matched so the motion will not show a match up. Blender will not do any morphing automatically to fix the mismatch. You could do an artistic/manual morph at the nearest point but that would be tedious and very noticeable, something akin to a jump cut in film editing. I believe that Disney has worked out some software that will do what you want but it is probably not available to the public at this time (I remember seeing a demo on the web of this and it was quite amazing.)
This is just my opinion. I too use blender as a compositing tool for film work, i.e. I am not a 3D artist.
Thanks for the reply, dancerchris. It’s all hand-held.
Below is an excerpt from the article which got me thinking along these lines. It’s discussing the long takes from the film Children of Men. I don’t have Maya or Shake so I’m trying to figure out if something similar can be done using Blender and After Effects.
“During rehearsals, our vfx editor Andy Hague would take the live feed from the video assist into editing software Final Cut Pro, and create test transitions directly on set. Using this reference, the camera move and choreography were then adjusted to provide the best transition points possible. Since the takes were all hand-held, we filmed the set with video cameras to see precisely where both the actor and the cameraman stopped at the end of each take. We then referred to these images to place them in the same position at the beginning of the next take. As soon as we had two plates, we would test the transition again and determine if we had what we needed to create a seamless blend before moving on to the next shot.”
The transitions were mainly completed in Maya using a 2-1/2D reprojection technique. First, the end of A-roll and the beginning of B-roll were tracked in 3D. A third camera was then created and used to blend the two camera moves. The 3D data generated by the three cameras was then exported back into Shake via proprietary software. This allowed the compositor to take the 3D data from the three cameras, and complete the physical merging of the plates within the compositing software, enabling him to retain maximum image quality from the original plates. The team often added foreground elements over the transitions to help with continuity issues and positional inconsistencies."
Thanks for the reply, dancerchris. It’s all hand-held.
Below is an excerpt from the article which got me thinking along these lines. It’s discussing the long takes from the film Children of Men. I don’t have Maya or Shake so I’m trying to figure out if something similar can be done using Blender and After Effects.
“During rehearsals, our vfx editor Andy Hague would take the live feed from the video assist into editing software Final Cut Pro, and create test transitions directly on set. Using this reference, the camera move and choreography were then adjusted to provide the best transition points possible. Since the takes were all hand-held, we filmed the set with video cameras to see precisely where both the actor and the cameraman stopped at the end of each take. We then referred to these images to place them in the same position at the beginning of the next take. As soon as we had two plates, we would test the transition again and determine if we had what we needed to create a seamless blend before moving on to the next shot.”
The transitions were mainly completed in Maya using a 2-1/2D reprojection technique. First, the end of A-roll and the beginning of B-roll were tracked in 3D. A third camera was then created and used to blend the two camera moves. The 3D data generated by the three cameras was then exported back into Shake via proprietary software. This allowed the compositor to take the 3D data from the three cameras, and complete the physical merging of the plates within the compositing software, enabling him to retain maximum image quality from the original plates. The team often added foreground elements over the transitions to help with continuity issues and positional inconsistencies."
Thanks for the reply, dancerchris. It’s all hand-held.
Below is an excerpt from the article which got me thinking along these lines (http://www.awn.com/vfxworld/children-men-invisible-vfx-future-decay). It’s discussing the long takes from the film Children of Men. I don’t have Maya or Shake so I’m trying to figure out if something similar can be done using Blender and After Effects.
“During rehearsals, our vfx editor Andy Hague would take the live feed from the video assist into editing software Final Cut Pro, and create test transitions directly on set. Using this reference, the camera move and choreography were then adjusted to provide the best transition points possible. Since the takes were all hand-held, we filmed the set with video cameras to see precisely where both the actor and the cameraman stopped at the end of each take. We then referred to these images to place them in the same position at the beginning of the next take. As soon as we had two plates, we would test the transition again and determine if we had what we needed to create a seamless blend before moving on to the next shot.”
The transitions were mainly completed in Maya using a 2-1/2D reprojection technique. First, the end of A-roll and the beginning of B-roll were tracked in 3D. A third camera was then created and used to blend the two camera moves. The 3D data generated by the three cameras was then exported back into Shake via proprietary software. This allowed the compositor to take the 3D data from the three cameras, and complete the physical merging of the plates within the compositing software, enabling him to retain maximum image quality from the original plates. The team often added foreground elements over the transitions to help with continuity issues and positional inconsistencies."
As mentioned there is no real morphing although you can do uv mapping on a plane and distort that. Even though you could match up single frames the real issue is continuing motion from one shot to the next. Which means stabilizing both shots, joining them, then reshooting with a virtual camera.
Theres a good reason for adding foreground element as they hide poor matches.
What you are trying to do usually requires a lot of prep work. On Children of Men, some of the long shots were faked, even when they contained rather shaky shots. As the two cameras will likely not share positions, you cannot simply cut from one to the other. What you will have to do depends on how the two cameras match up. If you can find a point where the cameras are almost at the same position, but have a different pitch and/or roll, you can create a plane and use project from view and texture baking from both cameras. Then you can use gimp or photoshop to blend the two textures. You can then create a camera motion that stitches between the two camera tracks. If you have moving foreground elements, you will have to roto them out first and composite them back in. Not sure how good that what work.
If the closest point betweem the two tracks is to far away, you can try the same trick as above, but you would have to do it in 2.5d.
It sounds like an interesting and challenging project.
Edit: facepalm, only just read your follow up with the quote about children of men, making most of my remarks redundant…
PhysicsGuy, I’m not very familiar with texture baking and blending. You’ve given me something to consider. I have a feeling I’m gonna end up having to do some camera mapping at some point.
Right now I’m having a heck of a time trying to setup two cameras with different tracking data into the 3D View. Every time I click “setup tracking scene” for one of my cameras, it changes the other. I know there’s a simple way to do this. Anybody?
The setup tracking scene is too simple for what you want. If you use setup tracking scene for the first camera, you should do the other camera by hand. This is done by simply adding a constraint on the second camera. At least, I think this is how it should work, but I have never used two cameras in one scene.
I’m really curious as to your progress. I think it is a really great thing to try. I have been playing around this evening, making a large panoramic texture from a panned shot. I baked a texture onto the inside of a cylinder by camera projection for every ten frames of the shot. Then I imported all those textures as layers in gimp and used the eraser to blend the layers together in on texture. This I then map onto the inside of the cylinder, so I can change the camera pan. This works really well. I think it would work for you as well!
Well, let me first say that it is not really my technique. I’m simply doing manually what Syntheyes does automatically (check the tutorial at http://www.youtube.com/watch?v=9MSLb4eND9k). The reason I’m experimenting with it, is exactly the example shown in that video: creating a
clean plate when the camera is not locked.
That said, I think one could use the technique to blend two videos, as in the example from Children of Men. If the position and orientation of the two cameras
is different, one would generate a 2.5D reconstruction use multiple time frame from both cameras to bake textures on that reconstruction. Then one can
make the third (fake) camera stitch between the two real cameras and to get rid of artifacts use a foreground elements (like the bus in the opening scene of Children of Men).
I think whether the techniques is convinving, depends on how well the person holding the camera is, because you need already a pretty good match between at least the position of the camera. I guess you can tolerate a little bit more difference in pan and pitch between the two cameras.
PhysicsGuy, I wish I had more to report. To be honest… I’m still in the early–trying to wrap my head around it–phase. My goal is to figure out the technique and run test shots before filming a short in April. My talent is writing. This is a challenge for me and not in my comfort zone. But, I know how I want the film to look and I’m determined to make it work.
I have access to professional steadicam gear and a couple of excellent DPs. I’m confident that I can get the shots. It’s the rest that I’m not so sure about.
Your comments are extremely helpful and appreciated. I wish that I had someone with your knowledge on my team. I’ll keep you updated as things progress.
I actually don’t have that much knowledge about this stuff, I just like to figure stuff out. Great to hear that it was useful. I’m actually thinking about writing a little script to automate baking the textures. I will post it as soon as I have something.
@Ugamic: It depends on what the camera is doing during the transition. If the camera is only changing orientation a little bit (which should be the case if you shoot the footage well), it shouldn’t make a difference. If there is a lot of pan, I would use the inside of a cylinder. If there is also changes in the pitch of the camera, I would use the inside of a sphere.
I think these are all secondary problems though. The bigger problem will be when there is a change in perspective during the transition. In that case, you need
to do a 2.5 or 3D reconstruction. After that, you can use the same projection tricks to texture the model.
This weekend, I will try to write a script to automate the texturing from view. I think it will be a useful tool as you can do camera mapping from your footage.