Hello I have a 4x4 model matrix `m`

that transforms an object from local space to world space. The matrix is in column major like this:

m00 m10 m20 m30

m01 m11 m21 m31

m02 m12 m22 m32

m03 m13 m23 m33

The matrix `m`

has initial value equal to `glm::mat4(1.0f)`

which is the identity matrix, and then it’s updated by rotation in this way: `m = glm::rotate(m, radians, axis);`

and I’m trying to get the basis vectors up, right and forward from it. How should I do that?

I thought this was a simple question at first but then ran into troubles. First, I thought I can take the 1st, 2nd, 3rd column directly as right, up and forward, but since OpenGL assumes a right-handed coordinate system, this is already wrong in the first step, because if z axis is forward and y axis is up, then the first column of identity matrix is “left” rather than “right”.

I know I can also compute the basis vectors by applying `m`

on the initial basis vectors, or using euler angles with `sin`

and `cos`

, for robustness, I finally decided to use quaternions and it does give me the correct result. However, I realized that some 3rd-party libraries only takes in matrix `m`

as parameter, and then assume that the first 3 columns correspond to right up and forward, I’m now using ImGuizmo to draw gizmos which does exactly that, as a result, the gizmo is not drawn correctly as I would expect.

Now of course there are ways to fix this, but I wonder if the way I use matrix `m`

is correct. Ideally, I want the first 3 columns of `m`

to be always right up and forward, but that would require me to initialize `m`

to a non-identity matrix. What should I do? Do I need to convert handedness of coordinate system?