Smart or AI Upscaling/Resize Open Source solutions?

I have seen that there are AI/Deep Learning/Smart solutions for this to work with games, videos or images. I am particularly interested in working with images, and Open Source solutions. I have found a couple of projects there:

But they seem complicated for end users. I would like to know end user options, download the ready-to-use software or, failing that, compile/build and use. Do you know any such option?

(I mean also, a software where you don’t have to run your own learning work/job/process on your machine, just a software ready to use)

2 Likes

I just found this one (waifu2x) that works on GPU, apparently using Vulkan?

There in Download section you can download the binaries for Windows, Linux and Mac. This by default upscale 2x. You have to work from the terminal/console. I suppose that by default it has a maximum of 2x because going beyond that would start to give not very good results.
There is explained how to work for Windows. For Linux it would be:
./waifu2x-ncnn-vulkan -i input.jpg -o output.png -n 2 -s 2

All possible parameters are described in the link. With the above upscale 2x with denoise level of 2.

Quick example:
*Original:
original

*4x GIMP - Cubic interpolation

*4x Waifu2x (Recursively 2x over first 2x pass)

Well, it is not magic but it is better than with Cubic interpolation on GIMP.

Have youtried the GIMP-ML addon for GIMP (currently for Mac and Linux only), one of the features is superresolution, along with other Machine Learning tools and filters.
(By the way, I’m pretty sure Waifu2x is geared more towards anime upscaling)

1 Like

Thank you!
Very interesting set of AI tools. downloading right now.
Here I have found some videos that describe them:

2 Likes

Hopefully they come to Windows as well, by the way, I’ve also heard that one of the GIMP developers (I believe his name is Ell) has plans to add machine learning to GIMP although I’m not sure for what (upscaling or selection…) although it is probably something that isn’t coming to GIMP any time soon

1 Like

Unfortunately, I have not been able to test it on Linux either. I’m having installation problems that other users have already reported. Let’s see how this progresses.

1 Like

Waifu2x works well with cartoon and anime, not detailed images. You can lost your details easily.

Apparently this is so, I guess it depends on the type of images that have been used for training. But I’ve tried some of my macro photos (where there is a lot of detail), and the result with waifu2x is noticeably better than common interpolation methods in GIMP. It is something that deserves to be tried at least.
4X example:

*GIMP:
gimp
*waifu2x:
waifu2x

Your all details gone and smudged. This works like xBRz algorithm and works good with cartoon images.

If you are Ok for this result, you can use ImageResizer.

Yes probably. Anyway that is why I have opened the thread. Hoping to try GIMP-ML addon to see the result with it.

Thanks, but Linux user here.

Try Wine or Mono, maybe will work.

@14AUDDIN . GIMP-ML has been updated, simplifying the installation process and adding the ability to update weights from GIMP. I have been able to install it on Kubuntu 18.04. Super-resolution plugin is giving really strange results in some cases (renders and photos). Here an example with BMW scene, super-resolution produce some artifacts on edges here:
*Original
input

*4x Super-resolution from GIMP-ML

Anyway it will be fun to continue experimenting with GIMP-ML, it has other interesting plugins like deblur

I’m going to try it when I have more time. The last time I tried to use Wine it was a headache creating the environments for 32 and 64 bits.

This is small program, I am sure it will work.

I think there are issues caused by the render noise, have you tried applying a denoising filter like Intel Open Image or the built in Cycles denoiser?

It is true that there seems to be residual noise, but the render is with 30 square samples and OptiX denoiser.
I don’t think that’s the problem anyway. If it requires completely clean images for good predictions, it doesn’t seem useful to me either.
I have also tried with photos, and there are really bad results. There is probably somewhere to discuss this stuff on github or somewhere to see if it’s bugs or something like that.

If you want, you can post here a small image (whose license allows it and preferably with lots of details) and I will do the scaling, so that you can see the result. By the way, images of no more than 400x400px because the plugin is giving me CUDA out of memory errors. It is using a lot of vRAM to my understanding, 4GB GTX 960 here. I’m going to make a request that one can choose to work on CPU when that happens, currently it autodetects CUDA and gives no options.

1 Like

Has anyone tried Cupscale, it uses ESRGAN and provides a graphical user interface for the user:

I’d love to give it a try, but there’s no Linux version unfortunately.

1 Like

There’s this Google Colab Demo, maybe you can try it out

Thanks for the link, but that looks way too complex for my skills. Actually I’m not even sure what that site is about :smile:
I prefer standalone apps.

1 Like

All you have to do is click on run all and then upload images when the “Upload Images” cell starts running