Calculate “u” in camera coordinates glulookat

n calculating camera coordinates from world coordinates, how do we calculate u coordinate? I know it is normal(cross(up, zaxis)) but i am not able to get the logic/ maths behind it.

You want to define a frame of reference. That is, an origin offset (the eye position) and an orthonormal basis (3 orthogonal unit vectors).

First, let’s compute the basis (u,v,w):

  1. Compute a vector from center to eye and normalize it (call this w)
  2. Take your given up vector and normalize it (call this “v_temp”)
  3. Cross v_temp and w to get u.
  4. Now cross w and u to get v.

NOTE: v_temp and v “may” be the same. The reason for step 4 is that we want to handle the case where the provided up vector isn’t quite orthogonal but is merely in the general direction that they want the up vector.

Dark Helmet’s formula above matches the Man page for gluLookAt (which is wrong), but not the code (which is right). As it is, you will get an orthogonal but not normal (unit determinant) matrix.

To correct it:
Skip step 2 above (too soon to normalize).
Add step 3.5: normalize u after making with with v_t x w.

Excellent point! Thanks for the correction. In the case where the “up vector” isn’t already orthogonal, the cross product from #3 would leave you with a vector of magnitude < 1, which I missed.