I have 3 perpendicular 3D unit vectors. Imagining them rotate around a Point in space, I can see how they could represent an object’s orientation. However, I have no idea how to convert those vectors into Euler angles (or a rotation matrix, quaternion, etc.). Thinking about it, I don’t think you need 3 vectors to describe every possible orientation. As long as there is a forward and an upward directional vector, there should be no confusion about how the object would be oriented.

Anyone have an idea on how to calculate an orientation from two (or three) perpendicular vectors?

I have no idea how to convert those vectors into Euler angles (or a rotation matrix, quaternion, etc.).

You don’t convert basis vectors to Euler angles. In computer graphics, euler angle rotations are usually represented by concatenating 3 matrices to rotate an arbitrary basis about a reference basis. For instance, concatenating a 3D rotation matrix which rotates about the X, Y, and Z axis of a reference basis with some angle a, b, y (alpha, beta, gamma) can be represented by 3 ^ 3 combinations. One of them is simply X(a) * Y(b) * Z(y) * v_n - where v_n is the n-th basis vector of the basis to be rotated with n element of {x,y,z}.

[QUOTE=w00tguy123;1240003]I have 3 perpendicular 3D unit vectors.[/QUOTE] If you have 3 perpendicular unit vectors, then you have the transformation (or rotation) matrix that rotates one coordinate system to another. Those 3 vectors make the columns of the rotation matrix.

BTW, if you have two vectors V1 and V2 the cross product will give you the missing third basis vector V3: V1 x V2 = V3. Remember to think of the handedness here! This will be perpendicular to the plane spanned by V1 and V2 - the handedness will determine in which direction V3 will point. V1 and V2 need not be perpendicular for this. However, in computer graphics we love our orthonormal basis. If you have a forward vector and an helper vector (not necessarily the real up vector as int eh case of gluLookAt) you can easily compute an orthonormal basis representing R^3 with the cross-product like follows:

Let F be the forward and H be the helper vector which need not be perpendicular. Assuming a right-handed coordinate system you can calculat a perpendicular vector R (right) with

R = F x H

Now you can calculate the third vector U (up) with

U = R x F

Normalizing the vectors will yield an orthonormal basis which, like the others already mentioned, can be used as a 3x3 rotation matrix.

Is there anything special I need to do when I apply the rotation matrix? Right now I’m calling glMultMatrixf() to apply the rotation but the object doesn’t get oriented correctly. Looking from a Euler perspective, the yaw is correct, but the pitch and roll are all wrong.

The order I put the vector data in is:
column 1: leftward
column 2: upward
column 3: forward

I’ve tried mixing up the order and inverting vectors but nothing looks correct. I haven’t tried all possible orders/inversions but I feel like I’m doing something wrong.

EDIT: Problem solved. Vectors had to be inserted by row, not by column. Thanks for the help everyone!