Mango project: Denoise or not?

Often in VFX work, you need to remove noise artefacts or sensor grain from the video footage and replace it after compositing. So that the effect shot sits amoungst the non-effects shots.

Are there any candidtes for denoise/noise nodes or VSE pluggins on the horizon?

Or are there tasks that should be done external to Blender, like the audio editing?

Red® cameras have grain?

Haha, all cameras have grain. I guess if you scale the 5k image down you will eliminate most of it though. Still they could also be performing a cropping (digital zoom in) of the image, which would enhance the grain.

and Cycles doesn’t? :wink:

Yep I thought that too. Quick way to shorten Cycles render times is with a dynamic denoiser. So that you spend longer on a clean frame but generate a difference table for successive short rendered (noisy) frames, that you could apply over the rest of the shot.

Also I wonder if the Optical Flow technology (interpolating frames based on movement sampling) would shorten Cycles times too? Handy for Blender to have such a pluggin :wink:

Noise/Grain management is definitely needed for Mango! So if anyone has an ideas - please tell us! :slight_smile:

In AfterEffects there is an effect that analyzes the footage and creates a similar noise pattern, which definitely helps to integrate CG.
The cycles noise looks totally different than any camera noise / film-grain. So better denoise cycles renders at first and than add film-grain to them. After that eventually denoise the mix of both. Well, I am talking out of the blue here, so that workflow might be total crap. :smiley:

Are there any open libs for that type of effect?

The Foundry has some awesome tools for denoising footage, I tried it recently on a 4K RED sample shot and it worked beautifully; granted it slow everything down to a crawl - but you can disable it while working.

I’m not really sure how it worked internally, all I know is that you have to select an area of footage that is flat (no detail), grey’s, and blacks work the best and you basically select an area and it computes that patch; the bigger the patch, the better the result.

The rectangle masking objects from the new tile-branch would be excellent for sampling areas - after that I suspect it’ll become rather challenging, I’ve no idea how they work. Do they take an average of the background colour, and the high-gain / noise and then figure out what percentage of blur to apply, and to what degree for each colour channel?

I can’t remember what channel the RED has the most noise on, but I suspect that it’ll be the blue channel, and if you could average out the noise of that channel in order to mute it, then you could probably get the values from that result and invert them to apply a full-screen noise effect at the end of the comp. setup.

P.s. It would be handy to have paint tools in the compositing window to paint over frames (and have reference frames in order to paint & hold patches over several dozen frames… matched up by tracking data of course for camera movement).

Well if Mango is using R3D files they are RAW (as close as you get from mosaiced chip) there should not be any more noise in the blue than any other channel (I don’t think that the chroma is subsampled in R3D).

I guess the alternative to denosing would be to add noise to cg renders (That’s usually what I do when doing composing stuff in Photoshop). Or maybe even some smart algorithm to analyze the noise pattern and make cylces recreate it…

sebastian_k: Denoise both Cycles renders and Red Footage. Do all the compositing. In the end add a little film grain (the latter maybe from Gimp, it got some nice options for grain).

Edit: Maybe create a plugin-node that can load gimp plugins/filters?

In real life, there’s noise everywhere in the shot because it’s noise from the camera, not noise from photons. With cycles, there’s only noise in areas that are hard to calculate, and solid parts of the image are too clean. Therefore, I recommend first removing noise from VFX, and then adding a bit of noise that mimics the uniform noise from real life footage.

I have often wondered why we can’t have fx pluggins from elsewhere? Is it a license issue? There are so many great open source pluggins around but I guess they all need maintainers.

Yep temporal noise removal, essentially any modern codec does that already, just to compress the signal. Maybe we should just render a low bit rate h264 as an intermediate NR filter :wink:

ha ha…this has just made my day…
…and I’m about to go to bed really so that’s a pretty darn good measure of awesomeness! :)))