DuctTake: Spatiotemporal Video Compositing

Maybe an interesting project I came across for the tracker/compositer tools of Blender:

DuctTake is a system designed to enable practical compositing of multiple takes of a scene into a single video. Current industry solutions are based around object segmentation, a hard problem that requires extensive manual input and cleanup, making compositing an expensive part of the film-making process. Our method instead compos- ites shots together by finding optimal spatiotemporal seams using motion-compensated 3D graph cuts through the video volume. We describe in detail the required components, decisions, and new techniques that together make a usable, interactive tool for compositing HD video, paying special attention to running time and performance of each section. We validate our approach by presenting a wide variety of examples and by comparing result quality and creation time to composites made by professional artists using current state-of-the-art tools.


http://zurich.disneyresearch.com/~owang/pub/ducttake.html

Except that Blender doesn’t do any temporal processing. Not even noise reduction or flow motion slow mo. So I doubt that temporal volumes (while awesome) are likely to be wedged into code :frowning: .

Anyway most of these examples are straightforward to produce with tracker and masks. Especially with new tracker distortion masks, coming soon from Sergey.

The restabilising stuff, where the image is distorted to maintain registration between shots is very cool though. No simple way to reproduce that.

Yes, I remember how you solved with at hand held shot of a boy disappearing into a forest. Apparently none of those included examples took more than 15 minutes to solve. It seems to be a very effective method.

I realize this is a bit off topic, but do you have a link for that? I hadn’t heard of this, and would like to read more about what is planned.

Thanks.
-LswaN

commit log – http://lists.blender.org/pipermail/bf-blender-cvs/2013-May/056086.html
example video – http://download.blender.org/ftp/incoming/mask_tracking_test.avi

Thanks. I guess maybe I misunderstood what was meant by “distortion” masks, as I had heard of this feature (though I hadn’t seen the video yet, pretty cool).

Btw, DuctTake looks really nice, and it would be nice to see something like this integrated into Blender. I think that anything to improve the roto workflow is welcome. The mask tools are pretty awesome and easy to use, but a method for automagically doing this stuff instead of pushing mask points around would be great.