OpenGL fog and shaders

OpenGL
1

I think I remember I read somewhere the OpenGl fog was a postprocessing operation that shouldn’t be replaced by the shaders. My tests seem to show the opposite (enabling a shaders removes the OpenGl fog).

Is there a way to have regular OpenGl fog and a fragment shader at the same time or do I have to include it in my shader?

Thanks

2

You have to do it in your shader. It’s simple enough.

Typically, when you do it in the shader, you might as well do it based on distance between eye and fragment, rather than plain Z depth. That’ll look better when you turn around, or look down on a flat plain from above.

3

It’s a quite interesting question, because there is very few example (in fact none) with fog and GLSL.

I’ve done some fog using ARB fragment/vertex program once.

Each time, you need to compute your Z fog value in the vertex and use it in the fragment program

It’s quite annoying because you cannot do fog in OpenGL with vertex shaders on video card that doesn’t have GL_ARB_fragment_program - Or at least, I’ve never managed to do so -

For GLSL, here some couple of functions that come in my mind (untested).

You should try and improves:

Vertex program:

float FogEyePlane(float4 Rh)
{
float4 Re = Rh / Rh.w;
float z = -Re.z;
return clamp((z - gl_Fog.start) * gl_Fog.scale, 0.0, 1.0);
}

float FogEyeAbsolute(float4 Rh)
{
float4 Re = Rh / Rh.w;
float z = abs(Re.z);
return clamp((z - gl_Fog.start) * gl_Fog.scale, 0.0, 1.0);
}

float FogEyeRadial(float4 Rh)
{
float4 Re = Rh / Rh.w;
float z = length(Re);
return clamp((z - gl_Fog.start) * gl_Fog.scale, 0.0, 1.0);
}

// And add in the vertex program section (parameter is the vertex in the view space).

gl_FogCoord = FogEyePlane(gl_ModelViewMatrix * gl_Vertex);

Fragment program:

gl_FragColor = mix(color, gl_Fog.color, gl_FogCoord);

There is also a gl_FogFragCoord. Never tried it.

4

Here a more ‘compilable’ version of the previous post + addition.

float FogExp(vec4 Rh)
{
vec4 Re = Rh / Rh.w;
float d = -Re.z;
return clamp( 1.0 / exp(d * gl_Fog.density), 0.0, 1.0);
}

float FogExp2(vec4 Rh)
{
vec4 Re = Rh / Rh.w;
float d = Re.z;
return clamp( 1.0 / exp(d * d * gl_Fog.density * gl_Fog.density ), 0.0, 1.0);
}

float FogEyePlane(vec4 Rh)
{
vec4 Re = Rh / Rh.w;
float d = -Re.z;
return clamp((d - gl_Fog.start) * gl_Fog.scale, 0.0, 1.0);
}

float FogEyeAbsolute(vec4 Rh)
{
vec4 Re = Rh / Rh.w;
float d = abs(Re.z);
return clamp((d - gl_Fog.start) * gl_Fog.scale, 0.0, 1.0);
}

float FogEyeRadial(vec4 Rh)
{
vec4 Re = Rh / Rh.w;
float d = length(Re);
return clamp((d - gl_Fog.start) * gl_Fog.scale, 0.0, 1.0);
}

void main()
{
gl_Position = ftransform();
gl_FogCoord = FogEyePlane(gl_ModelViewMatrix * gl_Vertex);
}

Fragment program:

void main()
{
vec4 color = vec4(1.0, 1.0, 1.0, 1.0);
gl_FragColor = mix(color, gl_Fog.color, gl_FogCoord);
}

5

That’s what I was affraid of. Not that it’s a big deal, but it would have been nice not to redo something that already exist.

Thanks.

6

Yes, you need to rewrite all the fixed pipeline for the fog in GLSL (as in ARB vertex/fragment too).

The advantage is that for example, you can implements the ‘radial fog’ on ATI, which is only works for nVidia in the fixed pipeline, or other custom fog model.

7

There is also a gl_FogFragCoord. Never tried it.
The gl_FogCoord built in is an input to the vertex shader, you should not be able to write to it. The GLSL spec defines gl_FogCoord like so:

attribute float gl_FogCoord;

and gl_FogFragCoord like this:

varying float gl_FogFragCoord;

You need to compute the fog coordinate in your vertex shader and output it to gl_FogFragCoord, like so:

vec4 eyePos = gl_ModelViewMatrix * gl_Vertex;
gl_FogFragCoord = abs(eyePos.z/eyePos.w);

Then in your fragment shader you would use the fog coordinate to compute a fog factor which you use to lerp between the fog color and your fragment color:

float fogFactor = 1.0;

if (fogMode == FogExp)
    fogFactor = exp(-gl_Fog.density * gl_FogFragCoord);
else if (fogMode == FogExp2)
    fogFactor = exp(-pow((gl_Fog.density * gl_FogFragCoord), 2.0));
else if (fogMode == FogLinear)
    fogFactor = (gl_Fog.end - gl_FogFragCoord) * gl_Fog.scale;

fogFactor = clamp(fogFactor, 0.0, 1.0);

gl_FragColor = mix(gl_Fog.color, gl_Color, fogFactor);
8

Also you must be very very cautious when using fog and GLSL.
ATI implementation is quite buggy (even in Cat 4.10).
Basically, you can end up with a ‘GL vertex shader will in software - unsupported language element used’.

This is what i found by doing some tests.

9

An addition : It’s simple, when you enable fog glEnable(GL_FOG), GLSL will work in software on ATI. End of story.

10

Originally posted by execom_rt:
GLSL will work in software on ATI.
What do you mean? The shader wouldn’t be executed on the GPU? That seams odd… But I haven’t tried on an ATI yet.

11

Originally posted by execom_rt:
An addition : It’s simple, when you enable fog glEnable(GL_FOG), GLSL will work in software on ATI. End of story.
I don’t see this behavior on my system. I know there was a GL_FOG bug before that caused software rendering but I believe it was fixed some time ago.

12

Well i mean when you enable fog, the GLSL shader said it won’t run in hardware and run slowly.

I’ve tried it on an ATI 9800 pro with Cat 4.10.

Yes it sounds like a old bug but seems to be still there :confused:

After some tests it seems to happens only on the first shader that it’s tried to be compiled:

For example

  • enable Fog.

  • compile and link first shader, linker says that it will run in software renderer.

  • compile and link a second shader, linker says that it will run in hardware.

I also found an other bug in the ATI GLSL - when using a complex expression it’s running out of temporaries registers, although the same code works in DX HLSL, even in ps_1_1 profile, but I will explain this in a future post -

Btw, it would nice that RenderMonkey can control the fog parameters, Quite hard to write a shader that uses fog without the fog parameter controls.

13

I confirm the bug - in an other application - . The trick seems to enable fog after linking the shader. Just plain weird.

See this post:

http://www.opengl.org/discussion_boards/cgi_directory/ultimatebb.cgi?ubb=get_topic;f=11;t=000322

14

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.