My friend needs some help with this glsl code

Hmm, the outline works great, nice touch on the size of it, really helps.

But i do have issues with the ao color can’t you just rip that out? for what is it needed?

to change the color of the effect?!
black = vec3(0.0, 0.0, 0.0)

i know the color codes, what i meant was i want a red outline, while everything else stays the same color. As is now the red color influence the other object a lot. I also read the comments i know you said keep same as ambient color, but does it need to be? can it do without?

this was originally intended to be a cheap ao method, not only a outline filter.

with this you can control color (line 11)
not need to use blur script.

uniform sampler2D bgl_RenderedTexture;
uniform sampler2D bgl_DepthTexture;
 
//USER VARIABLES-----------------------------------------------

float edgeSize = 5.0;      // control shading effect.

bool linesOnly = false;     // show shading effect only.
bool externalBlur = false;   // store the result in alpha pass for later blur.

vec3 lineColor = vec3(1.0, 0.0, 0.0); // control the color of lines.

//---------------------------------------------------------------
 
vec2 texcoord = vec2(gl_TexCoord[0]).st;
 
uniform vec2 bgl_TextureCoordinateOffset[9];
 
vec4 prewitt(sampler2D image, vec2 coords)
{
    vec4 samp[9];
        vec4 border;
        vec4 texcol = texture(image, texcoord);
 
    for (int i = 0; i < 9; i++)
    {
        samp[i] = texture(image, gl_TexCoord[0].st + bgl_TextureCoordinateOffset[i]);
    }
 
    vec4 horizEdge = samp[2] + samp[5] + samp[8] - (samp[0] + samp[3] + samp[6]);
 
    vec4 vertEdge = samp[0] + samp[1] + samp[2] - (samp[6] + samp[7] + samp[8]);
 
        border.rgb = sqrt((horizEdge.rgb * horizEdge.rgb) + (vertEdge.rgb * vertEdge.rgb));
 
 
float br = smoothstep(.005/edgeSize, .01/edgeSize, border.r);
float bg = smoothstep(.005/edgeSize, .01/edgeSize, border.g);
float bb = smoothstep(.005/edgeSize, .01/edgeSize, border.b);
 
 
        texcol.rgb = vec3(mix(1., .0, max(max(br,bg),bb)));
       
 
return texcol;
}
 
void main(void){
    
    vec3 image = texture2D(bgl_RenderedTexture,texcoord).rgb;
    vec3 lines = prewitt(bgl_DepthTexture, texcoord).rgb;
    vec3 combo = mix(lineColor, image, lines);
    
    float bw = (lines.r+lines.g+lines.b)/3; // lines to black and white

	vec3 final = combo; // lines on image
	float alpha = 1.0;

	if (linesOnly){
	final = vec3(mix(vec3(1.0), lines, 1.0));
	}

	if (externalBlur){
	final = image;
    alpha = bw; // lines on alpha
	}
	
	gl_FragColor = vec4(final,alpha);	
    
      
}
2 Likes

Thank you murilo!

1 Like

This version almost functional, downside on some angle it will color the whole plane instead of edges only.

use lower edgeSize
should be 1.0

Even with edge size 1.0, if you almost look flat on it (say almost looking below monkey) it will still color the whole plane

yea that’s how it works.

1 Like

Allright, not a big issue, project is on an stuck angle so won’t show, but just global testing and it happend so thought to mention it.

Tho if i zoom in 1 tic, then it’s fine, zoom out 1 tic then whole plane gets colored, guess it’s to smal to detect an edge.

a lower edgeSize will prevent the lines to show on far distances
but you can go lower than 1.0. (0.1 - 1.0)

no that won’t work bececause then it won’t show if i look like rts style top down on an angle, for that i need it to be at 10 xD.

but that works great! so all is ok

#edit hmm seems the distance have heavy influence on it :frowning:

yea it has
its the problem mentioned by OP