Pan tilt Camera and augmented reality


I am doing an augmented reality application : I have a known real 3D object … a camera mount on an high end Pan tilt unit (very precise) is watching the 3D Object, I need to add augmented reality information on top of the live camera feed.

  • I have a calibration process : I know the focal, real center of the camera : for the projection matrix I use : glMatrixMode(GL_PROJECTION); glLoadMatrixf(m)
    (m contains the good matrix calculated using focal, center,…)

  • I have also the modelView matrix when Pan and Tilt is set to Zero : I works fine : I match closely virtual and reality …

  • Now I need to move the virtual camera according to the real pan tilt camera offset … I have something like this :



–> If Pan and Tilt is set to Zero : It works fine,
–> If I move camera along Tilt Axis : It works fine,
–> If I move camera along Pan Axis (Even with Tilt set to Zero) : I have a strange roll that appears … and It don’t match … The more I move along Pan, the more I am out …

I don’t understand what’s happen : Is the way I do not correct ? I don’t understand why it’s works well for tilt and not for pan …

The problem is that rotations matrices aren’t rotation matrices. They are orientation matrices. They don’t simply rotate points around. Each matrix specifies a transformation from one space (aka: coordinate system) to another, and subsequent matrices operate on vertices in the space after the previous transformation.

For example, when you say “glRotatef(panAngle,0,1,0);” you are specifying a change of orientation about the Y axis. But which Y axis? That is, if a matrix is a change from one space to another, what is the input space?

Well, the input space is defined by whatever matrices come before it in the matrix concatenation order (which is, in terms of code, after defining that matrix). So the Y axis being rotated about is the Y axis after the tilt is applied.

You have the following spaces:

  • model space.
  • world space. (defined by “CalibratedModelView”)
  • post-tilt space. (defined by glRotatef(tiltAngle,1,0,0):wink:
  • eye-space. (defined by glRotatef(panAngle,0,1,0):wink:

The Y axis you want to rotate around for panning is the world space Y axis. But the pan matrix doesn’t see world space. It only sees post-tilt space.

A more in-depth discussion of this issue and solutions is available.

Thanks for your answer … I will check this but my problem for now is that it don’t works even if tilt is set to Zero (no influence) when I move only the pan axis ?? and if move in the same way only the tilt axis it works … Does it make sense ?

some news … I have done some modification using now Quaternion to set pan and tilt, and it works if I find the calibratedModelView when camera is horizontal (no tilt in calibration Model View). in this case I can move pan and tilt simultaneously and it works fine… If I find the calibratedmodelView when the camera is tilted … the pan is not good … the tilt works…

Do I need to extract pan, tilt, roll angle from calibratedModelView and mix it with the camera pan and camera tilt ? I am a little bit lost :slight_smile: