shadow maps in ATI cards looks ugly

I have shadowmaps working nicely in Nvidia cards, and altough they are also workin in Ati cards, they look ugly.

I’m setting up my code on the ATI version like this :

glGenFramebuffersEXT( 1, &g_BIG_SHADOWframeBuffer );
glGenRenderbuffersEXT( 1, &g_BIG_SHADOWdepthRenderBuffer );
glBindRenderbufferEXT( GL_RENDERBUFFER_EXT, g_BIG_SHADOWdepthRenderBuffer );

glRenderbufferStorageEXT( GL_RENDERBUFFER_EXT, GL_DEPTH_COMPONENT, 2048, 2048 );

The texture is created like this :


The Nvidia version instead of GL_DEPTH_COMPONENT16 i’m using GL_DEPTH_COMPONENT24.
In Nvidia even using a depth of 16, the shadowmap looks fine, i can’t even spot a diference from the 24bit version, so i guess they just roll back to 24.

The results are the following :


On Nvidia:

On the ATI looks pixelated and ugly, do you guys know what am i doing wrong here ? Maybe some other way to setup the fbo, or generate the texture ??


You’re doing nothing wrong. NVidia has hardware accelerated PCF (percentage closer filtering) which smoothes out the shadow test results. Unfortunately they had a patent on that, so ATI wasn’t able to include it in their video cards (i think that restriction was lifted for future DX10 video cards). If you want to achieve the same results, you’ll have to do the filtering yourself in a pixel shader, at the cost of many more instructions.$


damn , i wasn’t aware of that
How exactly is PCF done ?
I’m doing this in the shader ( i don’t know if it’s PCF) , but it gives the result you see above on ATI cards., i see around on the net PCF 2x2, PCF 3x3, etc, but i can’t find any examples on how to do it.

vec4 shadowC = shadow2D(shadowMap, shadowUV);
shadowC += shadow2D(shadowMap, + vec3( rfactor,  rfactor, 0));
shadowC += shadow2D(shadowMap, + vec3( rfactor, -rfactor, 0));
shadowC += shadow2D(shadowMap, + vec3( rfactor, 0, 0));
shadowC += shadow2D(shadowMap, + vec3(-rfactor,  rfactor, 0));

shadowC = shadowC / 5.0;
shadowC = clamp(shadowC, 0.0, 1.0);


Bruno, apparently there are two different concepts called “PCF” (percentage closer filtering).

  1. On this page, they talk about the kernel size, just like your implementation (around fig. 9) :

  2. and here, they talk about doing a bilinear filtering on the 4 nearest boolean results, this is what you need :

Basically, 2) provides bilinear interpolation, and 1) provides shadow blur. Both are needed to get nice shadows :slight_smile:

To say it another way:

Regular bilinear filtering takes four texel samples and weights them for each fragment.

Shadow comparison compares a texel value against R and produces a pass/fail result. But what happens when you try to use bilinear filtering and shadow comparison at the same time?

Both Nvidia and ATI hardware support bilinear filtering of DEPTH_COMPONENT textures. But when shadow comparison is on, the behavior is different (which is OK per spec):

ATI performs shadow comparison after any weighting filtering, so you always end up with a pass/fail result.

NV performs shadow comparison per sample, before any weighting filtering, so you end up with a filtered (greyscale) pass/fail result (this is PCF.)

In other words, to get the same behavior on both vendors, set your texture filtering to GL_NEAREST. In the fragment shader, you can take a bunch of shadow2D samples and weigh them yourself.

Thanks ZbuffeR and arekkusu :slight_smile:

"In the fragment shader, you can take a bunch of shadow2D samples and weigh them yourself. "

How can i do this ?
I tried getting the W component of 4 shadow2D and dividing them by 4, but i get no luck.
What do you mean by weight the samples ?

You can try something like:

vec4 tc = shadowUV / shadowUV.w;
float stepsize = 1.0/512.0;
vec2 sstep = fract(tc.xy*512.0);
//Get four samples
vec4 sh;
sh.x = texture2D(shadowMap, tc.xy+vec2(0.0, 0.0)*stepsize);
sh.y = texture2D(shadowMap, tc.xy+vec2(1.0, 0.0)*stepsize);
sh.w = texture2D(shadowMap, tc.xy+vec2(0.0, 1.0)*stepsize);
sh.z = texture2D(shadowMap, tc.xy+vec2(1.0, 1.0)*stepsize);
sh = sh < tc.z ? 0.0 : 1.0;
//Bilinear filtering
vec2 vert = lerp(sh.xy, sh.wz, sstep.y);
float result = lerp(vert.x, vert.y, sstep.x);

If you want to blend many bilinear samples, you can replace two lerp with one dot:

vec2 istep = 1.0 - sstep;
vec4 m = vec4(istep.y*istep.x, istep.y*sstep.x, sstep.y*sstep.x, sstep.y*istep.x);
float result = dot(sh, m);

You lose 3 instructions when blending one sample, but when you make more, you win.

thanks, but what exactly i do with the “result” variable ?
I see you take samples directly from the shadowmap, but i don’t get it.
First i blur the shadowmap, and i get the blured color value to be used, and then i do this bilinear filtering with the color obtained from the bluring, or i do this directly on the shadowmap ?
I tryed something like this :

shadowColor.w = result;

But no deal…

anyone ??

The result is the filtered shadow map (hence only one component float). Use it as you would use your unfiltered shadow map.

Well, using the result like that is not working,
this is what i get using it directly :

I think it looks even worse than the original ati problematic image.
Is there anything wrong here, on how i’m doing this in the shader ??

vec4 shadowC = shadow2D(shadowMap, shadowUV);shadowC += shadow2D(shadowMap, + vec3( rfactor,  rfactor, 0));shadowC += shadow2D(shadowMap, + vec3( rfactor, -rfactor, 0));shadowC += shadow2D(shadowMap, + vec3( rfactor, 0, 0));shadowC += shadow2D(shadowMap, + vec3(-rfactor,  rfactor, 0));shadowC = shadowC / 5.0;shadowC = clamp(shadowC, 0.0, 1.0);

float sample1 = shadow2D(shadowMap, + vec3( 0,  0, 0)),
sample2 = shadow2D(shadowMap, + vec3( 0,  mapScale, 0)),
sample3 = shadow2D(shadowMap, + vec3( mapScale,  mapScale, 0)),
sample4 = shadow2D(shadowMap, + vec3( mapScale, 0, 0));

vec2 texel_coord = max( * r_factor - 0.5, 0.0);

vec2 factor = fract(texel_coord);

float resut =  (1.0-factor.t) * ((1.0-factor.s) * sample1 + factor.s * sample2) +
factor.t       * ((1.0-factor.s) * sample3 + factor.s * sample4);
resut = clamp(resut, 0.0, 1.0);
resut = lerp(resut,shadowColor,mapScale);	

gl_FragColor = mix(gl_Fog.color, resut, fogFactor); 


Yeah, it’s definitely your shader. I remember some older nVidia developer demos using shaders with bilinear texture sampling.
A basic filter would be to sample surrounding texels of the current shadow map texel, average them and use that as your shadow your map value.

Try something like this (not tested, based on a previous post):

vec4 tc = shadowUV / shadowUV.w;
// 1.0 / shadow map size
float stepsize = 1.0/512.0;
//Get four samples
float sh;
sh = texture2D(shadowMap, tc.xy+vec2(0.0, 0.0)*stepsize);
sh += texture2D(shadowMap, tc.xy+vec2(1.0, 0.0)*stepsize);
sh += texture2D(shadowMap, tc.xy+vec2(0.0, 1.0)*stepsize);
sh += texture2D(shadowMap, tc.xy+vec2(1.0, 1.0)*stepsize);
// Simple average (samples / numSamples)
sh *= 0.25;

EDIT: Some comments. To get real bilinear sampling you have to take the fraction of the current pixel into account when sampling the shadow map.