Google's New AI Photo Upscaling Tech is Jaw-Dropping


In a post titled “High Fidelity Image Generation Using Diffusion Models” published on the Google AI Blog (and spotted by DPR), Google researchers in the company’s Brain Team share about new breakthroughs they’ve made in image super-resolution.

2 Likes

What a time to be alive!

3 Likes

How does this compare to that neural image processing software by Topaz (which can already be purchased), and is this algorithm optimized for portraits or can it work with any possible scene (which includes fanciful and/or abstract ones)?

Perhaps image upscaling could be improved dramatically if you can have the actual combined image pass done at the reduced resolution, but have pass information like normals and albedo at the target resolution (so you have the same feature guides as in denoising, which are very important to have for usable results).

It was optimized for a particular kind of images. The shown images are faces, but they trained other variants too with other data.

The presented super resolution approach is literally memorizing how things are supposed to look and is incredibly good at using it. In general, this isn’t going to work.
For this to be practically useful in the way you are describing it, it would need to be trained on very specific data. Like the having rendered a lot of renders for Sprite Fright and to improve the iteration times, a super resolution neural network could be trained on that specific data.
Another approach where this could be used is for animations. For this sort of upscaling, there is simply not enough information, but if you take multiple frames (possibly rendered with some tricks), you can combine the information to get more details.