per pixel spotlight off - me is clueless


I am trying to implement a per pixel spotlight, but I have the problem that depending on the position of the viewer towards the spotlight source, the light seems to wander around.

Setting up the camera:

void SetCamera (fVector *vViewerPos, fMatrix *mViewerOrient)
glLoadIdentity ();
glMatrixMode (GL_MODELVIEW);
glPushMatrix ();
glLoadIdentity ();
//x compensates for the screen aspect ratio
//z zooms somewhat in
glScalef (0.75f, 1.0f, -0.421875);   
glMultMatrixf (mViewerOrient);
glTranslatef (-vViewerPos.x, -vViewerPos.y, -vViewerPos.z);

Vertex shader:

varying vec3 vertPos, lightPos, lightDir;
void main (void) 
gl_TexCoord [0] = gl_MultiTexCoord0;
vertPos = vec3 (gl_ModelViewMatrix * gl_Vertex);
gl_Position = ftransform();
gl_FrontColor = gl_Color;

Fragment shader:

uniform sampler2D btmTex;
varying vec3 vertPos;
void main (void) 
vec4 texColor = texture2D (btmTex, gl_TexCoord [0].xy);
vec3 spotColor;
vec3 lightVec = vertPos - vec3 (gl_LightSource [0].position);
float lightDist = length (lightVec);
float spotAngle = dot (normalize (gl_LightSource [0].spotDirection), lightVec / lightDist);
if (spotAngle < 0.75 /*cutOff*/)
   gl_FragColor = texColor * gl_Color;
else {
   float attenuation = min (100.0 / lightDist, 1.0);
   float brightness = pow (spotAngle * 1.1, 8.0) * attenuation;
   spotColor = max (vec3 (brightness, brightness, brightness), gl_Color.rgb);
   spotColor = min (spotColor, matColor.rgb);
   gl_FragColor = texColor * vec4 (spotColor, gl_Color.a);

The spotlight will decay only after a real long distance and is not additive, that’s intentional.

A few pics to illustrate the problem:

Apparently spotAngle is not computed correctly if the viewer’s forward vector differs from the light direction, but I don’t know how to fix this. I’d have expected that all values passed via gl_LightSource are properly transformed.

If I turn scaling off, the problem disappears. As I need the scaling to properly display the level, the question is: How do I compensate for it when computing the spotlight?

Maybe I got it, partly. If I multiply vertPos.x and lightPos.x with the X aspect and vertPos.z and lightPos.z with the zoom factor it seems to work. There’s still some vertical movement of the light spot, although y doesn’t get scaled … ?

Adjustment for aspect ratio should be done in your projection matrix, not your modelview. Setting it in the modelview will affect your lighting calculations.

Of course, you’re right.

You know, this game (Descent) did all the matrix stuff by itself, and I am changing all this by and by, but it’s a lot of new stuff for me.