Hi, excuse me if this is the wrong forum category to post about beginner questions(I am a new to this forum), but my doubts are specifically related to GLSL coding.
I had found and since then worked upon a cartoon outline fragment shader for my game, which acts upon an offscreen buffer which contains a texture of the current camera view(a common techinic in games for many effects); the original code is at the end of post #26 on the following link if you want to check how the code was like( Blenderartists cartoon shader thread )
But what really matters is the code as it is now:
/*
* Toon Lines shader by Jose I. Romero (cyborg_ar)
* Updated, modified and commented by felipearts on 18/08/2011
* Based on blender's built-in "prewitt" filter which is free software
* released under the terms of the GNU General Public License version 2
*
* The original code is (c) Blender Foundation.
*/
//Performance Tips: If possible use * and +(in that order) in the same calculation instead of / and -;
//use dot product and avoid unecessary calculations or info splitting
//(Ex: for a vec4 calculation use split.abcd instead of split.abc and split.d)
#version 120 //BGE GLSL version present when this shader was written
//get external info
uniform sampler2D bgl_RenderedTexture;
uniform sampler2D bgl_DepthTexture;
uniform vec2 bgl_TextureCoordinateOffset[9];
//constant variables representating the fragment distance from camera
//can use a custom uniform from a python file in edgeForce too
const float near = 0.100;
const float far = 100.0;
const float edgeThresh = 0.25;
const float edgeForce = 0.6;
//a custom function, similar to texture2D
float depth(in vec2 coord)
{
vec4 depth = texture2D(bgl_DepthTexture, coord);
return -near / ((-1.0+float(depth)) * ((far-near)/far));
}
//the fragment shader loop
void main(void)
{
//assign these variables now because they will be used next
vec4 sample[9];
vec4 texcol = texture2D(bgl_RenderedTexture, gl_TexCoord[0].st);
float pixZ = gl_FragCoord.z / gl_FragCoord.w;
//gets all neighboring fragments colors
for (int i = 0; i < 9; i++)
{
sample[i] = vec4(depth(gl_TexCoord[0].st + bgl_TextureCoordinateOffset[i]));
}
// The result fragment sample matrix is as below, where x is the current fragment(4)
// 0 1 2
// 3 x 5
// 6 7 8
//From all the neighbor fragments gets the one with the greatest and lowest colors
//in a pair so a subtract can be made later. The check is huge, but GLSL built-in functions
//are optimized for the GPU
vec4 areaMx = max(sample[0], max(sample[1], max(sample[2], max(sample[3], max(sample[5], max(sample[6], max(sample[7], sample [8])))))));
vec4 areaMn = min(sample[0], min(sample[1], min(sample[2], min(sample[3], min(sample[5], min(sample[6], min(sample[7], sample [8])))))));
//The dot below is the same as a sum of the areaMx - areaMn result RGB components, but is more GPU efficient.
//The result is the average difference amount of the RGB(note alpha was left alone) components group
//of the two select fragment samples above.
float colDifForce = ((dot(vec3(areaMx - areaMn), vec3(1)))/0.5);
//Check for heavy RGB difference to darken the current fragment;
//we do not want to mess with transparency, so leave alpha alone
//edgeForce can be changed below to make outline more transparent or opaque
// ? : is the same as if else
colDifForce > edgeThresh ? gl_FragColor = vec4(vec3(texcol*edgeForce), 1.0) : gl_FragColor = vec4(texcol);
}
As you can see I could understand a good portion of it, and made quite an improvement to the original code; The current code does produce the desired outlining if the object in question is near the camera, but the more the object gets far from the camera, as expected, the outline starts being applied to undesired spots, as this fragment shader works upon the fragment depth, and the depth differences between fragments increases as they are farther from the camera.
I then thought which the shader should too be sensitive to the fragment distance to the camera, to compensate the fragment depth at greater distances(operating at the colDifForce > edgeThresh part of the shader). It came to my knowledge about gl_FragCoord; and which the below code outputs the distance of the pixel to the camera:
float variable_name = gl_FragCoord.z / gl_FragCoord.w;
I understood which gl_FragCoord.z is gets the position of the fragment into the window z axis, but my first question is what does gl_FragCoord.w is supposed to represent? And why dividing that by the fragment z position results into the fragment distance to the camera?
Also I suppose the ‘near’ and ‘far’ variables from the code start(present since the original code) were meant by the original author to represent the camera start clipping value(where the drawing start) and the end clipping value(where nothing is draw), is my understanding correct about that?
Thanks for any advising into that, I will not bother you with additional questions until the ones above are answered first.
PS: Please explain the best you can, as I am not that experienced in GLSL.
