First let me explain. I’ve purchased Topaz AI gigapixel because it’s ingenious and i can’t go without it now. It’s an AI upscaler that’s very fast on the GPU. About 5 seconds to 200% upscale an entire image from about 2K to 4K and denoise/deblur it at the same time. The results are almost perfect with only occasional unwanted artifacts or textures.
The idea is can this kind of technology be integrated into cycles. Not just for upscaling as that can be done afterward,. but could it be ran behind the scenes to upscale say 150% and then bicubic downscale it back to the original resolution, for each tile as its rendering, kind of like that Nvidia DLSS/supersampling. For a sharper output with minimal extra calculation time, similar to the denoiser option in cycles?
There is already another user on here working on a deep learning denoiser which already is showing great results. DeepDenoiser for Cycles
yea i’m very excited about that. And i would love it if somehow an upscaler was built into that denoiser, so it would take each tile, upscale & denoise, and then downscale, for a sharper supersampled image as cycles renders. If you get what i mean.
Having a super resolution solution in Blender would be great. I don’t see a reason why it should be restricted to Cycles. If it was integrated into the node editors (compositor, texture) and the viewport, artists would be encouraged to experiment with it and would have the flexibility to come up with unexpected use cases.
Some obvious use cases are:
- Upscale single frame
- Increase the resolution of textures
- ~Upscale the viewport (Cycles and Eevee)
You mentioned that upscaling a single frame and shrinking it down again can produce a clearer image. If there was an integration in the compositor, you could easily achieve it in there.
Upscaling the viewport could be useful for screens with huge resolutions (Eevee already has something like this I think). But the upscaling would need to be really fast to be practically relevant. This may also depend on the scene. If it is expensive to render, upscaling it might become suitable.
If Blender had its own upscaling solution, there would also be some amazing future potential. It could e.g. be extended to work for animations. Even if it was only possible to use it for previsualization purposes, it would still be useful. When you are dealing with decision makers, they often don’t have the background to judge work which is not final. If we had the ability to render a work in progress faster, it would surely simplify the communication with them a lot!
To make sure there are no misunderstandings: Using upscaling for animations would be a research project and it is unknown whether it would work. This could be achieved by combining some neural network building blocks which have been extensively researched, so it would not require some fundamental research.
I agree. And i admit i’ve upscaled all of my low res textures using AI Gigapixel already. It’s amazing. it can add detail to textures whilst keeping them at the size if you downscale it back again. Also yes it can be very very fast, on my 1070 to upscale a final image of say 2560x1440 by 200% would take about 5 seconds. It would be nice to have it as a node, for textures, as an option in the compositor, and wherever else it could be useful. The only thing is it must cost a fair bit of money to develop and integrate it into Blender, so it’s no easy task. The blender development fund is going to need more support. And I want to add that what you’re doing with AI denoising is probably the most exciting thing I’ve ever anticipated in Blender or rendering in general.
Hey, what kind of picture did you try to scale up? I was thinking about some AI scaler aswell since it would save me a lot of rendering time. Did you try that on some complex scenes with a lot of small details (like archviz renderes)? Price is 100$ for perma license right that is not bad? Pity they don’t provide some trial so I could test it out myself.
They offer a 31 day free trial.
Oh I must have missed that, thank you! Will try that out.
Topaz has a months free trial, yea i tried it on absolutely everything. Archviz usually works fantastic and takes out some noise whilst adding detail. Don’t forget you can always scale it back down to the original for a nice sharper cleaner image. It’s not perfect in every scenario and for large upscales like 600% it will start to get blurry and unrealistic. But it still impressed me enough to purchase it. Don’t forget to look for a discount code so you can get some % off the price, i managed 15% off,.
I really wish adobe would implement such a technology in photoshop. The closest thing they have atm is preserve details 2.0 upscaling, but that is way behind the latest image enhancement technology…
Just check github, there’s loads of open source AI implementations for super resolution, even proceduraly generated texture’s (cool one that takes a small section of an image, then creates a larger scale version based from that input
Which one do you think is the most practically relevant one as of now (good tradeoff between quality and performance)?
This is one of the best ive tested recently, handles 8x increase very well.
super resolution - This one is quite good as suports cuda & cpu.
neural-enhance: https://github.com/alexjc/neural-enhance
Anime- waifu2x
This has been updated very recently, works great for NPR styles
Image-Super-Resolution
This is a super resolution and denoiser combined setup which suports multiple different algo’s to test
dcscn-super-resolution
Good results from this, A tensorflow implementation of “Fast and Accurate Image Super Resolution by Deep CNN with Skip Connection and Network in Network”, a deep learning based Single-Image Super-Resolution (SISR) model
Super-Resolution-using-Generative-Adversarial-Networks
not tried this, but should be worth a look
Handling Motion Blur in Multi-Frame Super-Resolution
http://www.cse.cuhk.edu.hk/~leojia/projects/mfsr/index.html
BRCN
A unofficial implementation of paper method that ‘Bidirectional Recurrent Convolutional Networks for Multi-Frame Super-Resolution’
VideoSuperResolution
A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow.
pytorch
Collection of Super-Resolution models via PyTorch
pytorch-SRResNet
Liked this one, but was quite a while back i tested
These are just a few
That’s based on the publication which is from my point of view best suited for practical purposes. Its architecture has been heavily optimized for super-resolution to get a surprisingly good performance.
The GAN based variant of it is mind blowing (as long as you don’t have to train it 
)
yeah if you look at my last post your see it at the top :), forgot to add it the other day