video 
This is, without a Doubt, the most convincing Cellshader that I have ever seen â I start to like Cellshading now! Great Job, abc! : )
thatâs just absolutely beautiful. really highlights the environment design better than most rendering techniques.
wow, amazing as all the other jobs that you had done, congratulations ABC!!!
Hi again cartoon shader seeking friends, I found some knowledge candies about this subject(both opengl and effect specific); and in case you missed the ones at post #20 I included them too:
http://www.opengl.org/documentation/glsl/
http://www.opengl.org/sdk/docs/tutorials/
And non official
http://nehe.gamedev.net/tutorial/cel_shading/25001/
http://www.gamedev.net/page/resources/_/reference/programming/140/lighting-and-shading/cel-shading-r1438
http://en.wikibooks.org/wiki/GLSL_Programming/Blender
http://prideout.net/blog/?p=22
http://zach.in.tu-clausthal.de/teaching/cg_literatur/glsl_tutorial/index.html
http://www.opengl.org/wiki/Main_Page
http://www.blender.org/documentation/blender_python_api_2_58_1/bgl.html
I have not read them in details, but I recognize they may be quite useful for us. Also if you want to share any additional knowledge about this subject we would be grateful
EDIT:
I have been thinking, if one can pass python variables to GLSL in BGE; so I do not need to have a texture channel to assign the objet color as object differentiator! I need just to add a property to each mesh I want the effect upon; but also, there is another way: I can get the mesh name and verify in it for a flag I am yet to invent to call my ink shaded objects(ex: i_body). The only needed texture channel which my ink_shader would now demand is one for an ink map texture override!
Look C.A.ligĂĄri; I found a variant of the Jose I.Romero fragment shader which works upon depth buffer depth texture, which represents the depth of each fragment in relation to the viewer. With my current GLSL knowledge I added comments to it, version number and modified all the script below(and a few above) the sample for loop, seeking efficiency. There are still a inconsistency which were already present in the unmodified fragment shader(outlines appear in some undesired parts), but I think it could be worked out and the shader as a whole could be more efficient. It only gives outline for object edges, but it certainly is an improvement in reproducing my CG shader in GLSL, in a much better way.
My Version:
/*
* Toon Lines shader by Jose I. Romero (cyborg_ar)
* Updated, modified and commented by felipearts on 18/08/2011
* Based on blender's built-in "prewitt" filter which is free software
* released under the terms of the GNU General Public License version 2
*
* The original code is (c) Blender Foundation.
*/
//Performance Tips: If possible use * and +(in that order) in the same calculation instead of / and -;
//use dot product and avoid unecessary calculations or info splitting
//(Ex: for a vec4 calculation use split.abcd instead of split.abc and split.d)
#version 120 //BGE GLSL version present when this shader was written
//get external info
uniform sampler2D bgl_RenderedTexture;
uniform sampler2D bgl_DepthTexture;
uniform vec2 bgl_TextureCoordinateOffset[9];
//constant variables representating the fragment distance threshold as we want it for use next
//can use a custom uniform from a python file in edgeForce too
const float near = 0.1;
const float far = 30.0;
const float edgeThresh = 0.01;
const float edgeForce = 0.6;
//a custom function, similar to texture2D method, but instead use the depth texture provided
//by OpenGL(accessible with bgl) to know which rendered fragments are closer to camera and which are away
float depth(in vec2 coo)
{
vec4 depth = texture2D(bgl_DepthTexture, coo);
return -near / (-1.0+float(depth) * ((far-near)/far));
}
//the fragment shader loop
void main(void)
{
//assign these variables now because they will be used next
vec4 sample[9];
vec4 texcol = texture2D(bgl_RenderedTexture, gl_TexCoord[0].st);
//gets all neighboring fragments colors
for (int i = 0; i < 9; i++)
{
sample[i] = vec4(depth(gl_TexCoord[0].st + bgl_TextureCoordinateOffset[i]));
}
// The result fragment sample matrix is as below, where x is the current fragment(4)
// 0 1 2
// 3 x 5
// 6 7 8
//From all the neighbor fragments gets the one with the greatest and lowest colors
//in a pair so a subtract can be made later. The check is huge, but GLSL built-in functions
//are optimized for the GPU
vec4 areaMx = max(sample[0], max(sample[1], max(sample[2], max(sample[3], max(sample[5], max(sample[6], max(sample[7], sample [8])))))));
vec4 areaMn = min(sample[0], min(sample[1], min(sample[2], min(sample[3], min(sample[5], min(sample[6], min(sample[7], sample [8])))))));
//The dot below is the same as a sum of the areaMx - areaMn result RGB components, but is more GPU efficient.
//The result is the average difference amount of the RGB(note alpha was left alone) components group
//of the two select fragment samples above.
float colDifForce = ( ( dot( vec3(areaMx - areaMn), vec3(1) ) )/0.5 );
//Check for heavy RGB difference to darken the current fragment;
//we do not want to mess with transparency, so leave alpha alone
//edgeForce can be changed below to make outline more transparent or opaque
// ? : is the same as if else
colDifForce > edgeThresh ? gl_FragColor = vec4(vec3(texcol*edgeForce), 1.0) : gl_FragColor = vec4(texcol);
}
Original Martinsh code:
/**
* Toon Lines shader by Jose I. Romero (cyborg_ar)
*
* Based on blender's built-in "prewitt" filter which is free software
* released under the terms of the GNU General Public License version 2
*
* The original code is (c) Blender Foundation.
*/
uniform sampler2D bgl_RenderedTexture;
uniform sampler2D bgl_DepthTexture;
uniform vec2 bgl_TextureCoordinateOffset[9];
const float near = 0.1;
const float far = 30.0;
float depth(in vec2 coo)
{
vec4 depth = texture2D(bgl_DepthTexture, coo);
return -near / (-1.0+float(depth) * ((far-near)/far));
}
void main(void)
{
vec4 sample[9];
vec4 border;
vec4 texcol = texture2D(bgl_RenderedTexture, gl_TexCoord[0].st);
for (int i = 0; i < 9; i++)
{
sample[i] = vec4(depth(gl_TexCoord[0].st + bgl_TextureCoordinateOffset[i]));
}
vec4 horizEdge = sample[2] + sample[5] + sample[8] -
(sample[0] + sample[3] + sample[6]);
vec4 vertEdge = sample[0] + sample[1] + sample[2] -
(sample[6] + sample[7] + sample[8]);
border.rgb = sqrt((horizEdge.rgb * horizEdge.rgb) +
(vertEdge.rgb * vertEdge.rgb));
if (border.r > 0.8||border.g > 0.8||border.b > 0.8){
gl_FragColor = vec4(0.0, 0.0, 0.0, 1.0);
}else{
gl_FragColor.rgb = texcol.rgb;
gl_FragColor.a = 1.0;
}
}
It seems the BGE cannot do the technic I knew from panda3d, which involved offscreen texture buffers and multiple cameras; but maybe I do not need that, as it seems I can do it all with only a fragment shader(maybe a custom vertex shader too). If I can make somehow the shader give outline to each separated object(providing internal accurate edges too), and not a group of all of them(which gives only outer outlines of a character, both cloths and body); it seems GLSL have some functions for that.
EDIT: Hopefully fixed all the code comments
Iâve created a reel using this edge shader. Rendered with the BGE!
Amazing job abc, really incredible, can you share this filter with us with your setup on it? Its really incredible.
The models are extremelly creative, I loved the modullar vehicle!
@felipearts:
Interesting and useful â though to change the Opacity, the Value to be changed is ÂťedgeForceÂŤ.
My most proper Values are:
const float near = 0.1;
const float far = 1.0;
const float edgeForce = 0.5;
@abc:
Wheel Vehicle & Modular Vehicle⌠WOAH! xO
Wanna play that Gâââme! : D
Oh, sorry C.A.ligĂĄri; there was other comments with problems, but I think I solved that now(I hope); also I only know about colDifForce which if you set it to 0.0 the effect will give distorted outlines, but other values do nothing; I see a optimization possibility there. We need edge size control too.
If you take away all the depth texture stuff in my version and use standard texture samples as previously; the remaining setup I provided should work well with colDifForce(which would really work) to control the amount of edges, but then the fragment shader will take in account also the color differences inside your character, including lighting ones.
Also uh, sorry for post #19, I have not saw which you already explained how your gl_FragColor modification worked.
Hey abc123, which shader you was using there? The standard texture sample check(from the first posts) or a combination of it with the fragment shader variation I showed above(you already knew it? If no I would fell quite good for helping)? Also, have you made any custom improvements yourself? Are you still using the 2D filter logic brick/python or have you gone shader.setSource() and replaced even the vertex shader? Also, for your gui questions, know which moguriâs bgui isnât affected by 2d Filters, and it is a lot easier to create a gui with it; but keep in mind it is Beta yet.
EDIT: Added text content and better readability
Ok, unexplainably I started messing with my final custom variables at gl_FragColor for my custom sharpen filter and guess what? Now I can control the line numbers and size. Also the sharpen filter wasnât so hard to set up using my previous ink shader as base; the difference is which it tries to mimic contrast(sharp difference between dark and light) with his lines; below is the code. I will update my previous ink shader too. I am yet to understand the details of what I have done, but it is now improved, and it works!
/*
* Sharpen WIP shader by felipearts, 18/08/2011
* Based on Jose I.Romero Toon Lines shader modification by felipearts.
*/
//Performance Tips: If possible use * and +(in that order) in the same calculation instead of / and -;
//use dot product and avoid unecessary calculations or info splitting
//(Ex: for a vec4 calculation use split.abcd instead of split.abc and split.d)
#version 120 //BGE GLSL version present when this shader was written
//get external info
uniform sampler2D bgl_RenderedTexture;
uniform vec2 bgl_TextureCoordinateOffset[9];
//constant variables representating the edge settings
//can use a custom uniform for these too
const float edgeThresh = 0.5;
const float edgeSize = 0.9;
//the fragment shader loop
void main(void)
{
//assign these variables now because they will be used next
vec4 sample[9];
vec4 texcol = texture2D(bgl_RenderedTexture, gl_TexCoord[0].st);
//gets all neighboring fragments colors
for (int i = 0; i < 9; i++)
{
sample[i] = vec4(texture2D(bgl_RenderedTexture, gl_TexCoord[0].st + bgl_TextureCoordinateOffset[i]));
}
// The result fragment sample matrix is as below, where x is the current fragment(4)
// 0 1 2
// 3 x 5
// 6 7 8
//From all the neighbor fragments gets the one with the greatest and lowest colors
//in a pair so a subtract can be made later. The check is huge, but GLSL built-in functions
//are optimized for the GPU
vec4 areaMx = max(sample[0], max(sample[1], max(sample[2], max(sample[3], max(sample[5], max(sample[6], max(sample[7], sample [8])))))));
vec4 areaMn = min(sample[0], min(sample[1], min(sample[2], min(sample[3], min(sample[5], min(sample[6], min(sample[7], sample [8])))))));
//The dot below is the same as a sum of the areaMx - areaMn result RGB components, but is more GPU efficient.
//The result is the average difference amount of the RGB(note alpha was left alone) components group
//of the two select fragment samples above.
float colDifForce = ((dot(vec3(areaMx - areaMn), vec3(1)))/0.5);
//Check for heavy RGB difference to darken the current fragment;
//we do not want to mess with transparency, so leave alpha alone
//The variables at the shader start control the outline settings.
// ? : is the same as if else
colDifForce > edgeThresh ? gl_FragColor = vec4(vec3(texcol/edgeSize), 1.0) : gl_FragColor = vec4(texcol);
}
I think I have achieved quite a lot of what I wanted, I am satisfied for now; the nice thing is I learned more about GLSL, blender interface with it and about non photorealistic rendering. I will take a look at all the materials and links I have here now(yes, there are more than the ones I said before) to select the best ones for people interested in this. Now I think I will take a look at doing myself the bgui feature I need.
beautiful!
There is something at the beginning of the text does not work properly, if we add the ââ"no longer works
and even with # âŚ
a> â < in >blenderâs<brings up all red
just clean the â and all right :eyebrowlift:
/**
- Toon Lines shader by Jose I. Romero (cyborg_ar)
- Based on blender s built-in âprewittâ filter which is free software
- released under the terms of the GNU General Public License version 2
- The original code is (c) Blender Foundation.
*/
uniform sampler2D bgl_RenderedTexture;
uniform vec2 bgl_TextureCoordinateOffset[9];
void main(void)
{
vec4 sample[9];
vec4 border;
vec4 texcol = texture2D(bgl_RenderedTexture, gl_TexCoord[0].st);
for (int i = 0; i < 9; i++)
{
sample[i] = texture2D(bgl_RenderedTexture,
gl_TexCoord[0].st + bgl_TextureCoordinateOffset[i]);
}
vec4 horizEdge = sample[2] + sample[5] + sample[8] -
(sample[0] + sample[3] + sample[6]);
vec4 vertEdge = sample[0] + sample[1] + sample[2] -
(sample[6] + sample[7] + sample[8]);
border.rgb = sqrt((horizEdge.rgb * horizEdge.rgb) +
(vertEdge.rgb * vertEdge.rgb));
if (border.r > 0.2||border.g >0.3||border.b > 0.3){
gl_FragColor.rgb = 0.0;
gl_FragColor.a = 1.0;
}else{
gl_FragColor.rgb = texcol.rgb;
gl_FragColor.a = 1.0;
}
}
Allright, that âsatisfied for nowâ I said before passed; I came to the conclusion which the shader I modified from martinsh does not work well if the camera is set to perspective mode, and if this camera moves away from the desired outlined object, it outlines every small differences in depth the more the object gets far from the camera, I also noticed 3 things:
1 - Which this issue gets worse or better depending of the camera clipstart and clipend values, it seems the shader variables near and far should have each the same respective values which clipstart and clipend has(CONFIRMED).
2 - The lower the clipstart/near value, it seems the more inaccurate the shader outline gets(CONFIRMED)
3 - Any change to the the clipstart/near and clipend/far values makes the edgeThresh variable need a different value, which might indicate which additional calculations for this variable is needed .(CONFIRMED, additional math needed probably at last shader line)
Below is an image showing the issue:
I think the first step is to figure out how to send a variable to the shader(the camera start and end clip values), if this shader is a BGE filter one instead of a material shader. I also Private messaged martinish(since he wrote the initial code and has better GLSL knowledge), but Iâm not sure if I will get an answer.
EDIT:
I gone ahead and discovered which this is what I need to make the shader sensitive to pixel distance to camera:
http://www.opengl.org/sdk/docs/manglsl/xhtml/gl_FragCoord.xml
http://www.ozone3d.net/tutorials/glsl_fog/p04.php
EDIT 2:
I tested it, and it works with the current BGE GLSL version, this knowledge also unlocks the feature of regulating the line size based on the distance of the pixel from the camera.
EDIT 3:
I joined the OpenGL forum, Iâm getting there knowledge I could not got anywhere else. Recommended if you need help with complex shaders.
It turned out which gl_FragCoord couldnât be used, as a 2D filter operates into a flat screen quad, the z value returned by gl_FragCoord will always be the same. Ed Daenar at the official GLSL forum told me which I should rely only at the depth texture to achieve what I wanted. I like to share any improvement to the shader I made instead of keeping them just to myself, so here is how it is currently(Better/more comments, improved performance and simplicity and almost fully functional):
/*
* Toon Lines shader by Luiz Felipe M. Pereira(felipearts)
* Based on Toon Lines shader by Jose I. Romero (cyborg_ar)
* released under the terms of the GNU General Public License version 2
* updated 09/11/11
*
* The original code is (c) Blender Foundation.
*/
//Performance Tips: If possible use * and +(in that order) in the same calculation instead of / and -;
//use dot product and avoid unecessary calculations or info splitting
//(Ex: for a vec4 calculation use split.abcd instead of split.abc and split.d)
#version 120 //BGE GLSL version present when this shader was written
uniform float near; // The camera clipstart value, know in GLSL as near plane
uniform float far; // The camera clipend value, know in GLSL as far plane
uniform sampler2D bgl_RenderedTexture; // Gets the offscreen texture representating the current camera view contents
uniform sampler2D bgl_DepthTexture; // Gets the offscreen texture representating the current fragments depth
uniform vec2 bgl_TextureCoordinateOffset[9];
const float edgeForce = 0.6; // The force of the outline blackness
const float baseThresh = 0.001; // The initial(near value) edge threshold value for inking
// A custom function, which returns the linearized depth value of the a given point in the depth texture,
// linearization seems to be a way of having more uniform values for the fragments far from the camera;
// as it is logical which greater depth values would give greater results, also linearization compensates
// lack of accurancy for fragments distant to the camera, as by default a lot of accurancy is allocated to
// fragments near the camera.
float LinearizeDepth(in float z)
{
return (2.0 * near) / (far + near - z * (far - near));
}
// The fragment shader loop
void main(void)
{
// Assign these variables now because they will be used next
float sample[9];
vec4 texcol = texture2D(bgl_RenderedTexture, gl_TexCoord[0].st);
// Gets all neighboring fragments depths stored into the depth texture
for (int i = 0; i < 9; i++)
{
sample[i] = LinearizeDepth( float( texture2D(bgl_DepthTexture, gl_TexCoord[0].st + bgl_TextureCoordinateOffset[i]) ) );
}
// The result fragment sample matrix is as below, where x is the current fragment(4)
// 0 1 2
// 3 x 5
// 6 7 8
// From all the neighbor fragments gets the one with the greatest and lowest depths and place them
// into two variables so a subtract can be made later. The check is huge, but GLSL built-in functions
// are optimized for the GPU
float areaMx = max(sample[0], max(sample[1], max(sample[2], max(sample[3], max(sample[5], max(sample[6], max(sample[7], sample [8])))))));
float areaMn = min(sample[0], min(sample[1], min(sample[2], min(sample[3], min(sample[5], min(sample[6], min(sample[7], sample [8])))))));
float colDifForce = areaMx - areaMn; // Gets the average value between the maximum and minimum depths
float edgeThresh = ( sample[4] * baseThresh ) / near; // Limit value for the fragment to be outlined
//Check for heavy depth difference to darken the current fragment;
//we do not want to mess with transparency, so leave alpha alone
//edgeForce variable control the outline transparency, so 1.0 would be full black.
// ? : is the same as if else
colDifForce > edgeThresh ? gl_FragColor = vec4(vec3(texcol*edgeForce), 1.0) : gl_FragColor = vec4(texcol);
}
For that to work you will need for the object trigerring the filter2D to have two properties called far and near(thanks SolarLune for that), and you should through python let these two variables know the camera clipstart and clipend at the filter2D start, and update these variables if the camera clipstart and clipend values change, below is an example of how to do that:
own['near'] = logic.getCurrentScene().active_camera.near
own['far'] = logic.getCurrentScene().active_camera.far
Still thinking about the edgeThresh math, as I already know it is there which the final piece for a fully functional outliner would go(maybe edgeDifForce too or only one of them). Also I was right, the lower the camera clipstart value, the more the shader loses precision, it is something know in GLSL, and also the possible cause of planes flickering which happen sometimes.
Done, thanks to GLSL forum Ed Daenar again(and to myself of course), it seems to be fully functional, just further use of this shader at different scenes will confirm that. Now I will be taking a look at dynamically adjustable outline width.
/*
* Toon Lines shader by Luiz Felipe M. Pereira(felipearts)
* Based on Toon Lines shader by Jose I. Romero (cyborg_ar)
* released under the terms of the GNU General Public License version 2
* updated 09/11/11
*
* The original code is (c) Blender Foundation.
*/
//Performance Tips: If possible use * and +(in that order) in the same calculation instead of / and -;
//use dot product and avoid unecessary calculations or info splitting
//(Ex: for a vec4 calculation use split.abcd instead of split.abc and split.d)
#version 120 //BGE GLSL version present when this shader was written
uniform float near; // The camera clipstart value, know in GLSL as near plane
uniform float far; // The camera clipend value, know in GLSL as far plane
uniform sampler2D bgl_RenderedTexture; // Gets the offscreen texture representating the current camera view contents
uniform sampler2D bgl_DepthTexture; // Gets the offscreen texture representating the current fragments depth
uniform vec2 bgl_TextureCoordinateOffset[9];
const float edgeForce = 0.6; // The force of the outline blackness
const float baseThresh = 0.001; // The initial(near value) edge threshold value for inking
// A custom function, which returns the linearized depth value of the a given point in the depth texture,
// linearization seems to be a way of having more uniform values for the fragments far from the camera;
// as it is logical which greater depth values would give greater results, also linearization compensates
// lack of accurancy for fragments distant to the camera, as by default a lot of accurancy is allocated to
// fragments near the camera.
float LinearizeDepth(in float z)
{
return (2.0 * near) / (far + near - z * (far - near));
}
// The fragment shader loop
void main(void)
{
// Assign these variables now because they will be used next
float sample[9];
vec4 texcol = texture2D(bgl_RenderedTexture, gl_TexCoord[0].st);
// Current fragment depth
float base = LinearizeDepth( float( texture2D(bgl_DepthTexture, gl_TexCoord[0].st) ) );
float colDifForce = 0.0;
// Gets all neighboring fragments depths stored into the depth texture
for (int i = 0; i < 9; i++)
{
sample[i] = LinearizeDepth( float( texture2D(bgl_DepthTexture, gl_TexCoord[0].st + bgl_TextureCoordinateOffset[i]) ) );
colDifForce += base - sample[i];
}
// The result fragment sample matrix is as below, where x is the current fragment(4)
// 0 1 2
// 3 x 5
// 6 7 8
// From all the neighbor fragments gets the one with the greatest and lowest depths and place them
// into two variables so a subtract can be made later. The check is huge, but GLSL built-in functions
// are optimized for the GPU
//float areaMx = max(sample[0], max(sample[1], max(sample[2], max(sample[3], max(sample[5], max(sample[6], max(sample[7], sample [8])))))));
//float areaMn = min(sample[0], min(sample[1], min(sample[2], min(sample[3], min(sample[5], min(sample[6], min(sample[7], sample [8])))))));
//float colDifForce = areaMx - areaMn; // Gets the average value between the maximum and minimum depths
//Check for heavy depth difference to darken the current fragment;
//we do not want to mess with transparency, so leave alpha alone
//edgeForce variable control the outline transparency, so 1.0 would be full black.
// ? : is the same as if else
// abs is short of absolute value, it tells to disconsider the negativity of a value if it exists
abs(colDifForce) > ( sample[4] * baseThresh ) / near ? gl_FragColor = vec4(vec3(texcol*edgeForce), 1.0) : gl_FragColor = vec4(texcol);
}
Donât forget which to use it you will need to do what has been said at the post above.