Camera tracking and CMOS sensors (GoPro)

Hi everyone!

I’m posting here in hope you have more experience in camera tracking.

I have a gopro camera, and i want to use it for a vfx project with fairly quick camera motions. Unfortunately the CMOS sensors make rolling shutter effect, what makes the reconstruction almost impossible.
Do you have any idea, how could i solve it?
I’ve heard that some commercial software like AE can remove it, but i don’t know how good the result is.

And what would be a better solution? Recording in fullHD at 60fps, so it makes smaller rolling shutter bending, or recording in 2.7k at 30fps, what gives more pixels to work with, but larger bending as well? The final result wouldn’t be fullHD, only a medium resolution, at 24 or 30 fps.
So can any software get the jello effect out of a footage perfectly (for tracking)?

I read here that the “LibMv” team is working on something like that, but i don’t know how it is progressing

I hope you can help me, and hopefully it’ll be a useful thread for trackers in the future :slight_smile:

Thank you very much in advance!

I would recommend recording at 60fps and trying to drop frames, keeping the ones without rolling, however I’m not sure how plausible this is.

After Effects and Premiere Pro have Rolling Shutter Repair. I have used the CS5 version in AE and it works very well.

Or you could have a go at fixing it manually:-

Thank you! I’m afraid of that dropping frames would make the movie a bit strange looking. But i’ll try it!

Wow, it looks cool! So you’re saying that a simple plug-in would solve it almost perfectly?
I’m on linux, but i’ll ask my friend if he can fix it in After Effects. If it works, we’ll stay at 2.7k, that’s far better quality.
That manual method could work in blender as well! I guess. Just some masking and layer compositing…
Thank you!

Ummm about AEffects and Rolling shutter repair… BEFORE you track a shot. You are adding to the problem when doing this effect. You are essentially changing the pixels to accommodate the warp from lines of info being recorded. Best to re shoot, FPS at 30 is fine. The less info to transfer to the card the better. 60fps not needed here. IF you were going to go Slow mo or 1/2 step your footage then 60 fps would play in.
IGE: same movement scenario and slowing time / remapping of footage.

Your case. What is your recordable format? DO you have JPG 2000 available? Anything with the least amount of compression is your friend in tracking. If you are losing the track re shoot with heavy contrast tracking points. IGE: Light on dark points. Black stickers on white movement. Then after track assign masks to points to cover up.

This is a case where a DSLR for shooting video is a good choice due to the record format of JPG2000 @ 24fps or 29 pend on broadcast out. Computer uses 30. FWIW.
Drop frame refers only to the way your NLE counts frame on your T.L. (in editing)

Thank you!

GoPro doesn’t have jpg2000 format, but it has a protune option. It gives a raw, neutral colour, and about 50 kb/s instead of the normal 30 kb/s (mp4-h.264). This way we the footage has less contrast, but it’s easy to adjust it later.

Artificial tracking points are a great idea, some litter on the ground would help us a little bit.