[GLM] Inaccurate camera rotations

I’m using the latest GLM in my little engine.

I’ve got the following bit of code to rotate a first person camera:

vec2 delta = 0.01f*(vec2)mouseDelta;

m_player->rotate( quat(vec3(0, delta.x, 0)) );
m_player->rotate( quat(vec3(delta.y, 0, 0)) );

This in turn calls:

void Transform::setRotation( const quat &rotation )
{
	m_rotation = rotation;
}

void Transform::rotate( const quat &rotation )
{
	setRotation( cross(m_rotation, rotation) );
}

Finally, the model and view matrices are computed:

void Transform::update( double deltaTime )
{
	mat4 rotationMatrix = glm::mat4_cast( normalize(m_rotation) );
	mat4 translationMatrix = glm::translate( mat4(), m_translation );
		
	m_modelMatrix = translationMatrix*rotationMatrix;

	m_viewMatrix = inverse( m_modelMatrix );
}

The matrices are uploaded and used by GLSL in the following way:

uniform Transform
{
        mat4 view;      // Transform::m_viewMatrix
        mat4 model;     // Transform::m_modelMatrix
} transform;

mat4 modelView = transform.view*transform.model;
mat4 modelViewProjection = projection.perspective*modelView;

gl_Position = modelViewProjection*in_vertexPos;

The camera properly rotates around its x or y axes if only one of the two calls to m_player->rotate() in the first snippet are left uncommented. If, however, I allow for rotations around both axes, the rotation becomes inaccurate; the rotation is then no longer limited to the x and y axes but is also slightly rotated around the z axis, tilting the camera to the left or right.

Am I doing somethign wrong here? Is there a way to prevent this?

Thank you!

quat(vec3(0, delta.x, 0))
looks wrong. Init quaternions by specifying an axis, and then angle. Right now you’re probably specifying just a direction.

The constructor of glm::quat takes a glm::vec3 which is suposed to use euler angles. It might be better to use glm::angleAxis to build your two independent axis
http://glm.g-truc.net/api-0.9.2/a00287.html#gaac770715e5e3dc93f48c6634b59efd43
This function is declared in <glm/gtx/quaternion.hpp>