I have been trying to solve this problem for a couple weeks now and I need some help. The problem I am having regardless of what system I use (Euler, Axis-Angle, Quaternion) is how to determine the appropriate future rotation from the changing local axis. For example:

Camera is at position (0,0,0) for simplicity. OUT is +z, UP is +y, RIGHT is +x. Rotation of the camera using arrow keys is relative to the screen at all times. As soon as I rotate I will not be aligned with the global axis anymore, so I have to assign x, y, and z rotation accordingly.

The confusion is something like this: I know for quaternions you can multiply Q1 * Q2 for example where Q1 is your current orientation and Q2 is a change to that orientation. So if I press the up arrow, how do I create Q2 to always be a rotation towards the top of the screen? The same issue applies to axis-angle format as well.

Do I need to store and manipulate the vector for each axis for each rotation or is there a more direct approach (can you “lose” your up vector or do all 4D vectors preserve the orientation)?

Isn’t the point of quaternions that you can do movement in multiple axis at the same time and arrive at the same orientation regardless of the order in which you applied them? Euler angles affect each other and cause gimbal lock due to their interaction.

The easiest type for me to visualize is axis-angle, but I don’t know how to do relative rotations for that as each axis changes. I imagine tracing the inside of a sphere where the XYZ tells me where I am and the angle tells me which direction is the relative “up” for example. It’s necessary to do all the rotations from the local perspective, but how?

I guess the issue is with all the things I’ve been reading, I know how to convert between the different types, to some degree how to multiply the current orientation by an offset, but not how to derive what I need to multiply into the current orientation based on my goals.