When I read OpenGL related documents, I saw something like this
normal matrix is the inverse of the upper left most 3 x 3 of gl_modelViewMatrix
In my understanding, the normals we specified is in object coordinate system. So what is the purpose of modelview inverse matrix? I am confused with its definition and possible usages. Anybody can help clarify?
Hmm, a little complicated.
The real reason for the inverse is that rotations and translations do not change the length of vectors, but scale operations do.
Non-uniform glScale parameters will squish your normals to point into the wrong directions. You need to use the inverse scaling to keep them pointing into the correct direction.
To visualize this take a sheet of paper and draw a circle with normals pointing outwards. Now scale one axis by 0.5 giving an ellipse.
Where do the normals point if you would scale them by 0.5 on the same axis?
See? Instead they need to be scaled by 2.0 on that axis.
Note, scale operations require the normals to be renormalized to get correct lighting. OpenGL does this for you with glEnable(GL_NORMALIZE).