Help with shadow map projection.

So, here’s what I’m doing:

Set projection matrix to light’s gluPerspective
Set modelview matrix to light’s gluLookAt

Render the scene geometry into a depth-only FBO

Set projection matrix to players’s gluPerspective
Set modelview matrix to players’s gluLookAt
Make texture matrix using:
(bias matrix) * (light’s projection) * (light’s modelview) * (camera’s modelview inverse)

Render scene properly using the following shadow coordinates specified in the vert shader:

shadowCoord = gl_TextureMatrix[3] * gl_ModelViewMatrix * gl_Vertex + vec4(0.0,0.0,-0.004,0.0);

the offset is there to prevent z-fighting.
What is supposed to happen is the camera’s modelview inverse in the texture matrix should cancel out the camera transformation in the gl_ModelViewMatrix in the vert shader leaving only the object transformation (I think).

This is what happens:

However, if I place the gluLookAt for both the light and scene camera into the projection matrix, shadows are rendered correctly (but lighting breaks):

Am I missing out on a step? The drawing of the geometry includes calling glLightfv with GL_POSITION, but taking it out didn’t help.

I’m not sure what other details might be requried, but here’s the main bits.

Note: I also posted this on GameDev last week.


First you should check whether camera’s modelview inverse is correct by pre-/postmultiplying it with gl_ModelViewMatrix and checking if it returns the identity matrix.

I assume you’re using a projective texture lookup instead of a 2D texture lookup?


I think you confuse you matrices here, it’s never good to try and cancel one thing out when it’s not nessecary.
what i do is that i combine only the light’s projection and modelview matrices and put it in the texture matrix.

then i just combine it with the coordinates in the vertex shader
gl_TexCoord[2] = gl_TextureMatrix[0]*gl_Vertex;

then in the fragment shader i transform it to fit the shadow textures space

gl_TexCoord[2] = gl_TexCoord[2]/gl_TexCoord[2].w;
gl_TexCoord[2]=(gl_TexCoord[2]+ 1.0) * 0.5;

and now i only have to compare it with the shadow texture like this
if((shadow.z+0.005) < gl_TexCoord[2].z)

read more here

Yup, inverse modelview * modelview has been checked to return the identity. (I transpose the rotation and negatise the translation for the inverse from my recollection)

is what I use in the frag shader.

How do you get the vertex’s world-space coordinates, though? gl_Vertex contains the values from the vert buffer/array/glVertex, which are relative to the object’s position. I can see this approach working when all object are drawn at the origin, but not otherwise.

Thanks for the replies guys :slight_smile:

That’s only valid for the identity rotation matrix.

Modelview = | R3x3 t3x1 |
            |   0    1  |

Inverse   = | R3X3_transpose -R3X3_transpose*t3x1 |
            |        0               1            |


Wow, thanks, I guess when I checked that the inverse was working I only looked at the rotation matrix (the numbers had very low orders of magnitude), I kind of assumed it would work given the vector maths library I’m using isn’t written by me.

Again, thanks, this problem has been a serious pain in the ass for the past week :smiley:

Yea sorry about that, i code to fast for my own good sometimes, though it’s a quick fix to add that.

  1. add the inverse matrix to the texture matrix
  2. instead of this
    gl_TexCoord[2] = gl_TextureMatrix[0]*gl_Vertex;
    i now have
    gl_TexCoord[2] = gl_TextureMatrix[0]gl_ModelViewMatrixgl_Vertex;

thanks Zeoverlord.

Your comments were helpful to me