Yes, its just that there are different ways / methods to describe and find noise.
There is specific info in the other layers, but if we can see it in the end result, a neural net should be able to detect it too.
Or should at least be able to some noise level based blur for pixels alike it, with help of some pre calculated stats.
It is all about your quality requirements.
I am pretty sure, a big enough neural network can detect even more noises than we can do, just by looking at the final image. But do you have a quantum computer with petabytes of RAM and maybe 100 years of patience to wait for the training?
We need the passes to cut-off the complexity. More than that, we may need better (new kind of) passes to even surpass Disney’s results, I have seen some interesting options in other softwares like Substance Painter.
FYI: Talking about training time, AlphaGo Zero trained on a desktop PC with a single GTX 1080 Ti would take at least 50 years to get the same results that DeepMind has claimed.
@CpuRender, in his earlier trials RasorBlade seamed convinced a small network could do it.
Not a behemoth network based upon google’s hardware. AlphaGo Zero’s neural network was trained using TensorFlow, with 64 GPU workers and 19 CPU parameter servers, not a single GTX1080 as you describe. I think RasorBlade was one of the first here to do something with neural networks here at BA. Apparently he makes a living out of AI / robotic vision, so maybe the guy deserves some credit.
Anyway you each have different ways to get there, both of you want a few animation movies.
CpuRender wants, EXR data of all shader layers.
Razor only wants low res and high res png’s
Maybe its a good idea to start with the classroom scene, Rasors earlier work used it too.
So if someone is reading this thread with pretty good hardware.
Feel free to generate some data these guys can work on.
Notice classroom was also rendered in eevee (maybe thats the future of rendering?)
Not a behemoth network based upon google’s hardware. AlphaGo Zero’s neural network was trained using TensorFlow, with 64 GPU workers and 19 CPU parameter servers, not a single GTX1080 as you describe.
I was talking about the Reinforcement training supported by Google’s TPUs.
I think RasorBlade was one of the first here to do something with neural networks here at BA. Apparently he makes a living out of AI / robotic vision, so maybe the guy deserves some credit.
Did I discredit him in any way?
in his earlier trials RasorBlade seamed convinced a small network could do it…
CpuRender wants, EXR data of all shader layers.
Razor only wants low res and high res png’s
It’s getting even more interesting to try out both to compare.
Notice classroom was also rendered in eevee (maybe thats the future of rendering?)
Things will change with the NNs.
Any kind of comparison with AlphaGo (Zero) in the context with a denoiser makes absolutely no sense. It uses reinforcement learning while all current denoiser approaches rely on supervised learning, which is understood a lot better and requires a lost less training.
The future of rendering is more likely to use actual path tracing, with dedicated accelerations. I expect that it is even going to be used for realtime rendering in combination with e.g. SVGF. We are still a few years away for this to work in realtime and in order to achieve high end results, other and more computationally expensive techniques are most likely needed.
Referred to training time vs quality.
supervised learning
You can achieve more quality from longer training. And reinforcement learning also works in this case, it’s just cheaper to do it supervised at the moment.
There is nothing you can compare. If you train a simple supervised model for this amount of time, you are just wasting a tremendous amount of computational time, because it usually plateaus a lot faster and does not become better anymore. There is even the risk of overfitting. That’s why a comparison does not make sense.
After a certain amount of time, you reached the maximum quality and further training does not help anymore or may even produce worse results due to overfitting.
Of course, reinforcement learning could be used for denoising, but that’s not what we are talking about! And as such, it is worth nothing to make comparisons with it.
I wasn’t limiting the network’s size or architecture. Anyway, let’s wrap this up and wait for the first implementation.
NVidia recently released IRay 1.4 for Rhino with integrated real-time denoising
Looks like a great helper. Is this feature also considered for Cycles?
their denoiser seams to improve over time thats quite nice.
it also seams that they blur depth of field pretty fast, using distance as a factor for blur.
if that would be combined with adaptive rendering…wow
Wow, it’s sooo lovely that in-preview denoising!
Because of some problems at work i had a non productive workday (a day of waiting).
So i had time to play with my neural nets and i could spend quite some time on this.
My coding toolbox got expanded tot he point that i now could use a animation rendered as i described earlier.
So if someone could render a quality animation of the classroom alike the eevee demo, in .png format i be gratefully.
My own hardware would take month’s to render something like that in cycles.
The renderings i could use to train a neural net.
Can you be more specific about:
- what resolution, min. sampling, clamping, color mapping & how many frames are needed at least?
- can couple of different scenes or rendered using different engines help too?
&
- did you get any already?
Indeed very impressive 
@burnin.
It would be great to have something like the classroom scene.
At 1280x720 or so, normal simple render settings but it depends a bit on how you render (CPU/GPU), as for GPU i wouldnt know.
No clamping, Current denoiser disabled,
For CPU 32x32tiles, Normal PathTraching width 100 and 300 samples, would be input material (or a GPU quality alike it).
To train against a high sample rate 1000 or more.
But that high setting depends on the hardware you have, for me rendering like that would take month’s.
The higher the better i think.
For me i think 70 frames will do as minimum though 250 would allow me to do more testing in time-based denoise.
Rather saved as png file instead of a compressed AVI file (compressing adds noise).
The scene could be the class room, maybe later some more scenes, for validation to test the neural net against different situations.
Well so far i dont have any movie material to work on, the code still evolves a bit slowly though; i’m the kind of programmer that tends to think long about algoryhtms and then at some point start coding, i got several types of neural nets; and i’m thinking of mixing them together with some clasic image kernel operations.
If the denoiser is going to improve via the neural network/machine learning route, perhaps it would be best if it made use of the existing Cycles renderpass data as a guideline system to help create the right result (or at least get very close to how it would look with tons of samples).
That could reduce the level of training that would be needed and ameliorate the concern of overfitting (because the result would also need to fit the information in the passes).
ok, have made some small changes on the classroom scene (shaders mostly, to reduce fireflies)
Note, there’s no difference in image quality at same amount of samples on CPU or GPU.
Q: Static or animated noise seed?
@ 1000 samples, unbiased (no clamping) - variations:
-
a) lights ON, blinds ON
-
b) lights ON, blinds OFF
-
a) lights OFF, blinds ON
-
b) lights OFF, blinds OFF
& An animation previz (145 frames @ 25fps): [video]https://i.imgur.com/6Q4ldL2.mp4[/video]
If there’s no other comments, notes,… except for the noise seed, pick a variation & i’ll deliver asap.
@burnin 145 samples would be nice, i thought 1000 samples would give a noise free result.
Maybe your rendering GPU and so larger tiles, and that would require more samples ?.
From above i’d say it would be best to use 1b.
The others are interesting too, but it might be to noisy i think to be a target goal to solve for a neural network.
(the globe map on the wall is too random)
Although i’m often surprised what they “can learn”.
Maybe if possible do a 145/2 = ~ 72 samples of 1b, and 72 samples of a bit improved darker version ?.
Perhaps use light portals (not sure if its used) for the darker versions, if the current denoiser helps on the darker version then enable it on that one, keep lighter as is alike 1b.
Oh also animate the seed value, as the neural network should not learn the static sample distribution; as i once spotted that.
PS my plan would be to write a 2d Network might be single hidden layer still or multi, but first layer would consist of various areas from different frames
ea from
9x9 part of frame 4,
7x7 part of frame 3,
5x5 of frame 2
3x3 of frame 1
and let it solve center 1 pixel on frame 1, … as a kind of funnel, let the NN choose the best pixel based upon (likely) future neighbors.
Though i have some other plans to test as well above would probaply the first, after i’ve finished my GF kitchen repair / makeover :))
PS might sound as small number of pixels, but store HSL data (thats times 3) and i store some more statistical data as wel for input probaply, so it quickly counts up. maybe i will use even less pixels, i dont use Keras or so, i got my own training methods that works quite well (as i rather create networks who i can understand and improve then create huge ‘beasts’ of unknown magic, what seams pretty popular these days, i still prefer the old days of elegant code and smart tricks.
- Am working with dailies (render using GPU+CPU together).
- Yes, Light portal is added.
- Clamping was OFF, why so much noise is present. Clamping now ON.
But it darkens the result. Why i was assuming raw results are preferred (physically more accurate as more energy/light is distributed). eg.
-------------------------------------------------------------------------------------------------
Would it be useful if second part (darker version) is further split into 2 parts (40 frames each)? Will this do?
- LIGHT /Frames 001-080 @ 1000 samples (denoise OFF)
2. DARK / Frames 066-145
a) 066-105 @ 1000 samples (denoise ON - *splotches/flickering will occur in animation)
b) 106-145 @ 2000 samples (denoise OFF - animation will be grainy, but bit less flickery)
PS.
Motion Blur: ON or OFF?
@burnin
second part darker 2 times 40 is fine, …
In the end i will train the neural net like this.
I divide the frames in images, from which i take tiles, as an array of tile[x,y,frameN] so width 40 frames it would still mean i have lot of tiles to train from.
Maybe then keep denoise off, flicker behavior is not what i currently would be able to handle, maybe as input but not for a validation data set. (your creating a validation data set, ea the goal the neural network has to learn).
Motion blur would be OK ( in fact i rather have it ), as it doesn’t make it complex, but it is how animators usually render.
(should be on by default in blender i think)
Also for the camera motion, can you save the blend as well (with your motion), as i’d like to render low quality as well, basically that would be the input, while your rendering should be the output to train.
I had not time to code, i might need 2 weekend more to rebuild the kitchen, after that i hope i will have some time.
Or maybe i will have some luck at work sometimes i can code my neural nets there, as we use them there to (industrial/vision).