Denoising video with Blender blues

Long story short, my canon 550D sux big time at recording videos in low light conditions. Despite what the manuals, training videos, books, forum posts and everything else I’ve come across say, recording noise-free video in low light is (next to) impossible. Or should I say impossible for me with the standard kit lens (18-55mm)? Using a shutter speed of 50 (i.e. double the frame rate of 25p) and the widest possible aperture (3.5) I really have to go up to iso 1600 or more to get a half-decent exposure. The result is a lots of grain and a videos as noisy as the ones I used to shoot with my old SD camcoder.
I’ve ordered a new lens with a higher aperture (1.5) which should solve the problem (in theory). I’ve also installed magic lantern which is freaking awesome but I only found out about it a few days ago. The thing is that I cannot go in the past and reshoot an event which was held in improperly lit conditions in early December. So I’m stuck with a grainy video that needs heavy denoising before editing. Now, the million $ question is how does one actually denoise video in the compositor?

Some background research revealed a few interesting suggestions which had been posted on this thread:

The problem with the suggestions made in that thread is that they do not seem to work out of the box (or to be more precise I could not get them to work). Obviously, more work was required. Thus, I started doing various experiments. I ended up removing some of the noise but was not pleased with the result at all.

Then I considered using some other app to see if I could obtain better denoising. Eventually, I used Avidemux and two video filters cnr2 (chroma noise reduction) and hqdn3d (high quality noise reduction). Using both filters in combination removed most of the noise and I was very happy with the result. It was better than what I’ve been able to produce by a factor of 5 (or more).

At that point I run a few tests checking the number of unique colors and the scopes to see what was happening. This is a screenshot of the upper left corner of a frame from the original quicklime video. I guess I need not comment on the noise. The number of unique colors is quite high (about 300K) and all 3 scopes seem quite dense.

Here follows the same denoised frame from Avidemux. The number of colors has been reduced by a factor of 2(!), the histogram edges suggest changes, and the scopes look less dense. The difference in the vectorscope is quite telling. Now, if I trust my eyes and not the scopes and all, there is a big improvement and most of the noise is gone. Thus, this is an acceptable result.

Now the same frame after some color grading. Number of unique colors goes down even more, the histogram clearly reflects color manipulation, but the scopes are more dense compared to the denoised frame. In terms of visible perceptual differences, I think this is an improvement over the denoised one.

Finally, I’ve had the crazy idea of combining the noised source with the denoised one and tried that in the compositor. The result speaks for itself in terms of visual quality. Again, all three scopes are less dense.

[screenshot 4 to appear in a new post due to stupid attachment per post limitation]

While using Avidemux was cool and the two filters were totally awesome, I hate doing stuff I have little control over or stuff that I don’t quite understand. What’s more, I’d love to do all denoising within Blender.

How would you guys approach such problem? What’s the best method for denoising video in the compositor? Should I trust what I see or the scopes? How come and every color grading manipulation results in a less dense signal representation in the scopes? How can that be avoided?

Any tips/suggestions/workflows will be greatly appreciated.
Please think of this as a video-denosing brainstorming invitation.

Here’s the 4th screenshot (sorry for the second post but I’m not sure how to bypass the imposed restriction)

After reducing noise I often add some back in for apparent sharpness and dithering for brightness changes (during a mix etc.), just remmeber that the quality of the noise is important. Remove chroma noise add luma noise (B&W), anyway not sure how far you will get with pixel convolution in Blender, but you need to split out colour channels, blur (convolve if possible) and mask edges for detail preservation. Haven’t had much success but then I’m a hack.

Look at other node solutions with similar tool sets and try to replicate in Blender. I want to fix my 550D footage too.

Another idea, you could shoot HDR mode (magic lantern) without much ev change at 50 or 60 fps and add 2 frames for increased sensitivity and built in noise reduction. down side is loss of resolution at 720p.

But how do you reduce noise? Do you do that in Blender or do you use some other app?
Unfortunately, I couldn’t find documentation on how denoising is done in other sw packages. Most of the info I could find suggested using some After Effects denoise filter! :wink:
Other suggestions were more in the direction of avoiding noise when shooting in the first place (use better lenses, improve lightning conditions with artificial light etc). All these are fine but not usable for the task at hand for I already have shot the video.

Could you test the sample blend file on one of your noisy sources and tell me if/how it works?


denoising_sample.blend (623 KB)

Honestly, I try to avoid shooting it. Sorry. Because Blender doesn’t have straight forward NR.

Quite frankly I find it impossible to believe that no other Blender users face denoising issues with the videos they shoot.
I mean, does everyone shoot with the Red one or sth? :wink:

Well i thought that I could check out some old Photoshop techniques, as they often had to invent the wheel before a suitable pluggin was available.
Check this out fopr chroma NR:

Basically you split the chroma from the luma and blur it then add it back to the luma as a color mix.

How about a bit of dark frame subtraction. You will need a noise black sample from your camera’s sensor. Don’t know how repeatable it is, should be ok for fixed pattern noise.

In a similar vein, I have found that shooting magic lantern’s HDR and adding the light/dark frames tends to remove noise as different noise is mixed together.

Here is a good trick with various chroma splits. The drawback is a lack of median operation.
But probably doable in Blender. Really couldn’t you do all this with GIMP NR?

Hmmm, a shake median filter…

requires a math node…

You can save your video as image sequence and batch process with neat image or noise ninja. These programs are not expensive and cross platform.

Or try nuke. Its a big bag of wonders.

Even GIMP can do a batch NR. For free. But the point is doing EVERYTHING in Blender! :wink:

Anyway it’s handy if you just want to do one shot.

3point thanx for the pointers.
There are good news and bad news. The good news is that I’ve experimented a lot and have reduced the footage noise significantly. The bad news is that I’m not quite there (yet). Noise levels are down with some of the methods I used but still that doesn’t match the denoise filters of Avidemux.
Will start with each suggestion I’ve tried, share my experiences and post blend files with the noodles I used for others to consider.

This might work in theory but I found it very limited for my purposes. How do you create a Despeckle filter in the compositor anyway?
What’s more, the guy finds the solution acceptable for print but to my eyes it is still noisy as hell and, overall, rather unacceptable as a solution.

Sounds like a great idea but I’ve restricted myself to software solutions. Worth experimenting in the future though. Have grave doubts about how reproducible it is as a method.

Ya never know where the gems reside, could be under any old dragon. Thanks for testing. In the meantime I have been shooting the noisiest footage possible! Doubt that I will be able to fix it though. I will post a link when it is up.

This was a good read and gave me several ideas. I split image into YCbCr, sharpened the luminance and blurred the chrominance components. The effect was marginal but it led me to consider other directions. See attached blend files.
In one of them (first) I’ve managed to isolate the noise itself. The question is how to proceed. Nothing I’ve tried produced results. Any tips on how to combine it with anything else? (125 KB) (177 KB)

This was an excellent suggestion! Had two main problems with it. First, wtf is a reorder node? Second, what is a colorX node and how to you make one in the compositor? After some digging up, reordering was easy but implementing colorX without an expression node was very challenging. Maybe I’m wrong but I don’t think replicating this noodle from Shake is possible in the compositor.

Here’s what I’ve come up with by following his instructions: (187 KB)
That’s obviously a failure since the guy was able to get color and a denoised image while only I got a bw one! :0
I guess that even if one could find a way to connect the nodes as suggested in the post, it would still be impossible to replicate it in Blender. Getting proper input per channel and returning predictable output puzzled me.

A couple of questions. First, when he says “add” does he literally mean “add” or simply combine, e.g. mix? I’ve tried both and none works. Second, how does one combine the colorX node with the rest of the noodle? I’m kinda stuck at step 4 of the process described. Btw, it’s super cool what the guy does in one step in the exp node while I had to beat about the bush with several nodes. Any tips on more elegant solutions?

uh…if there’s anyone left still reading the thread and/or checking the blends posted, I’d appreciate any feedback.

I’m still here, but don’t have much time at work to check them out with my footage. Should be able to do this next week. Thanks for all the grunt work. Sorry I haven’t any better ideas, so many times I find that someone just uses a NR node out of the box! Frustrating.