Live Feed Oil Paint Filter

Hey guys, I have been working on this project for quite some time now and it has been an amazing learning experience. I am so excited to finally share the process. I am a studio art major currently in college and I have had to do quite a bit of traditional mediums since starting. Even though I have learned these mediums, my biggest passion is still 3D. 3D is a medium that my college doesn’t currently teach, but I still wanted to incorporate it into my projects whenever I can. My first attempt at this was by traditionally painting pbr textures and scanning them to be used in a 3D scene. This was a huge eye opener for me and made me want to push the idea of MIXING TRADITIONAL AND DIGITAL ART. My latest project I have been improving is a way to make a 3D Model look traditionally painted without the use of post processing or compositing. My first attempt at this worked very well! I started with a normal scene and would render it out in toon style shaders. I then “Composited” while still in a 3D scene by placing a camera in front of a plane with the render on windowed mode. I then would skew the plane’s vector to create different paint like effect. I then would render it again with an overhead light and paint normal maps. This is a couple of my results of using this method.

I then started wondering if there was a way though to make it like a live feed where I would no longer require multiple renders. I began with the thought, a camera to a texture. If I could just find a way to transfer a live feed of a camera view to a screen then I could make it work, but with a lot of searching I found no way of making this possible. Then after searching for a very long time I finally found a very strange way that almost worked like a camera filter. I found it on this link from 2 years ago. https://blender.stackexchange.com/questions/79749/zoom-window-magnification

This started me down a whole new project which surprisingly only took me a few days to figure out. This method in simple terms reflects a specific location of the cameras view onto a plane. In my scene instead of using the reflective plane to mirror only part of the scene I had it mirror the exact dimensions of the camera and reflect it back into the camera where I am able to skew the plane to give it the effects I wanted. This ended up giving me a painterly filter on the camera where everything could be moved around in the scene while still seeing the effects and not having to composite it. Here are some results of the filter.

image


This is huge in my eyes cause it opens the world to so much with effects that no longer have to be rendered in order to view them. I am very limited in this mode though. The results are not quite up to par with my old method which is why I came here. The issue is that by doing this reflective maneuvered with the normal input of the reflective node (See my project file, reflective node on “Oil Paint Filter” material). This makes it so that I can not add actual normal maps to it. I also cant give it a roughness effect to mimic paint like I could in my other method. I was able to kind of trick the normal map method by adding a vector normal node and connecting it with the color of the reflective shader. this meant I could add fake shadows to the reflective plane to act like a overhead light. I am still limited though because I cant find a way to make the highlights to mimic the roughness. I also am unable to cleanly rotate the paint effect like I could in the other method. In the old method I simply rotated the skew image, but this causes issues in my new mode. this makes it hard to use in animations without it looking like a cheap photoshop filter. So overall it isn’t as good as my original, but it is still amazing with how it works and I am wondering if anyone that is good with blender nodes could help me solve some of these issues. This is probably super confusing for most of ya’ll. I’m not very good with words, so I am just going to give ya’ll the project file and I hope it can explain things better. (note: this has to be used in 2.81 or above) This shared folder also includes some videos that show how it works. They are too large to upload on here.

https://1drv.ms/u/s!AkT_M95NR0GkkahjUTb3b8waSUJzxA?e=8g68Ov

11 Likes

A new update of the project. I was able to simplify the node tree. While doing this I was able to make the texture rotatable. This can be used for animations. I rendered this animation in under 20 min. (on my computer that is good)

20191219_020926987_iOS

https://1drv.ms/u/s!AkT_M95NR0GkkalNp9zOFoAtxeiV0w?e=eYPvCU

5 Likes

wonderful and very interesting

1 Like

Your study is very promising! Thank you very much for sharing!
Edit:
I’ve just seen the file and this is pretty awesome! Would it be technically possible to achieve something similar with Eevee?

1 Like

Didn’t see this reply until today. I have managed to do it, but it’s very impractical. Since it uses a method called “zoom magnifier” it requires proper reflections on a plane. In cycles when a reflection created on one side, you can still see it on the other side as well. so it acts like glass. In eevee this doesn’t happen though. It just shows black on one side while showing the reflection on the other. The only method I found that worked was to place the camera and plane backwards in the scene to only see the reflected side. This is where it becomes impractical since you can’t move the camera normally. It is very annoying haha. I think that there will be a way in the future, but currently, there isn’t,

:smile: well … On the s.n.e.s. Donkey Kong 3 we have some levels that the controls play on the opposite directions and I have managed to pass them… It would be uncanny and strange, but in practice animating backwards is just as valid as foreword :slight_smile:
Thanks for the explanation! It’s still a valid concept to me.

I got it to work properly in eevee by changing the IOR of the Refraction material to 2. It does enlarge the image slightly, but otherwise it works well. Great work on this shader btw, it’s really awesome!

1 Like

Looks like @Puffertron solved this. Welcome to Blender Artists Puffertron!

Here is another suggestion. Maybe using using a backward camera and plane and a forward camera would allow normal camera movement. Use the forward cam view position it in the scene while the backward cam and plane follow it and create the effect.

Parent the backward cam and plane to the forward cam so they move with it. Position these relative to each other so the reflected view is the same as the forward cam view. Look through and move the forward cam to arrange the view as needed. Have another work area (pane? panel?) showing the backward cam view for the real time eevee brushstroke effect.

A problem is that the forward cam would see a reverse image of what the backward cam sees. If there is a way to reverse a camera’s image then do that with the backward cam.

1 Like

@Alan_Wyatt
I admire your persistence in the technical side to look for aesthetics results. The fact that you’ve broken the limitation of the traditional setup (stylized oil painting) to translate it into the ray vectors of the camera, shows endless stylizing options.
At the moment I am working in a watercolor shader. You can follow our thread on here>>

I guess most of us are becoming truly free as artists to create using blender, which inspire others as well. Thank you for sharing your setup. I hope to see your new renders.

2 Likes

thanks a lot! It’s really fun putting stuff like this together.

1 Like

That didn’t even cross my mind! Great idea!!

Very cool! Ive seen people trying to accomplish similar things to what we are doing. My biggest issue is direction. Your idea of using vector smearing is brilliant. I never would’ve thought of that. I tried for the longest time to capture a “Flow map” of an object, but I never could get it to work how I wanted. The UV direction in the vector paint mode is a great idea. It might take time, but its a great way of thinking about it!

1 Like

Could you share your file for this? I’d like to see how you got it to work

Back face culling. I haven’t used this so it is just a guess. Maybe this would let you see through the plane in eevee as in cycles.

1 Like

Beautiful, i like the possibilities of this.

1 Like

I made some procedural shaders but didn’t know it would look similar to your filter.

So I voluntarily helped a person with a project and here is the first test render: https://youtu.be/oAs9jOX_IEA

There was only one of my materials was used in that test, and I haven’t receive any message from the person for a while.

Here is the first one, the ground shader: ground.blend (939.4 KB)

The second one is a wall and ceiling shader: wall.blend (616.1 KB)

The third one is an EEVEE stylized PBR shadow: shadow and cube.blend (926.3 KB)
The cube’s material is an example of mixing the effects.

The fourth one is an EEVEE stylized PBR glass shader: Glass.blend (738.2 KB)

The last one is a fishbone (chevron) shader: fishbone.blend (745.6 KB)
I made it for a duvet in a scene. I don’t have the image of the scene, but the idea is making the sleeper under it look like a dead fish.

2 Likes

Interesting. What is the goal of your shader? Are you trying to make it look like paint?

Yes. We wanted to make it look like this: https://youtu.be/wm3l4R-Is70
But using shader in 3d.
If I checked your filter earlier, it would be easier for me. Thanks for sharing!

I also have this.

https://gum.co/ewLrd.

It may be of more help then the oil paint filter.

2 Likes

Cool effect, but damn, did you never hear about something called line breaks and paragraphs?! Solid blocks of stream of consciousness text are very hard to read for people with dyslexia!