Double Vision and Camera Tracking

I am still trying to integrate OpenXR into an existing game engine. I am sitting OpenXR up to be used in a VR headset with a room-sized play area. Couple more issues I could use some advice on:

1: Double vision. The left and right eye views don’t line up. Looking into the headset at a single 3d cube, it is spaced too far apart. I am thinking this is an issue with the game engine’s default shaders and not a setup issue with OpenXR. Using Open GL ES.

2: Camera tracking. Looking left and right line up. Looking up and down line up, but if I turn 180 degrees, now looking up and
down become reversed. Also tilting my head, the ground tilts ahead of me, if that makes sense.

I am using XR_REFERENCE_SPACE_TYPE_STAGE with xrLocateViews during frame rendering.

I am only using XR_REFERENCE_SPACE_TYPE_STAGE and XR_REFERENCE_SPACE_TYPE_VIEW reference spaces. Should I be using XR_REFERENCE_SPACE_TYPE_LOCAL for anything?

Here is some sample code I am using to setup reference spaces and sitting the game engines camera.

void CreateReferenceSpace()
        {
            XR_MESSAGE("-- Create Ref Space: Start ----------------------------");
            if (getopenxrstatus() == Failed_Status) return;

            // Fill out an XrReferenceSpaceCreateInfo structure and create a reference XrSpace, specifying a Local space with an identity pose as the origin.

            // Create LOCAL Reference Space: Not using local ref space, not needed?
            XrReferenceSpaceCreateInfo referenceSpaceCI{XR_TYPE_REFERENCE_SPACE_CREATE_INFO};
            referenceSpaceCI.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_LOCAL;
            referenceSpaceCI.poseInReferenceSpace = {{0.0f, 0.0f, 0.0f, 1.0f}, {0.0f, 0.0f, 0.0f}};
            OPENXR_CHECK(xrCreateReferenceSpace(m_session, &referenceSpaceCI, &m_localSpace), "Failed to create Local ReferenceSpace.");

            // Create VIEW Reference Space
            XrReferenceSpaceCreateInfo viewSpaceCreateInfo = {XR_TYPE_REFERENCE_SPACE_CREATE_INFO};
            viewSpaceCreateInfo.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_VIEW;
            viewSpaceCreateInfo.poseInReferenceSpace = {{0.0f, 0.0f, 0.0f, 1.0f}, {0.0f, 0.0f, 0.0f}};
            OPENXR_CHECK(xrCreateReferenceSpace(m_session, &viewSpaceCreateInfo, &m_viewSpace), "Failed to create View ReferenceSpace.");

            // Create STAGE Reference Space
            XrReferenceSpaceCreateInfo stageSpaceCreateInfo = {XR_TYPE_REFERENCE_SPACE_CREATE_INFO};
            stageSpaceCreateInfo.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_STAGE;
            stageSpaceCreateInfo.poseInReferenceSpace = {{0.0f, 0.0f, 0.0f, 1.0f}, {0.0f, 0.0f, 0.0f}};
            OPENXR_CHECK(xrCreateReferenceSpace(m_session, &stageSpaceCreateInfo, &m_stageSpace), "Failed to create Stage ReferenceSpace.");;

            XR_MESSAGE("-- Create Ref Space: End ----------------------------");
        }
        void SetGameEnginesCameraPosition(float *X, float *Y, float *Z, XrTime predictedTime)
        {
            *X = 0;
            *Y = 0;
            *Z = 0;

            XrSpaceLocation headLocation{XR_TYPE_SPACE_LOCATION};

            XrResult result = xrLocateSpace(m_stageSpace, m_viewSpace, predictedTime, &headLocation);

            if (XR_SUCCEEDED(result))
            {
                if ((headLocation.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT)    != 0 && 
                    (headLocation.locationFlags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT) != 0 )
                {
                    XR_MESSAGE("Success: Head Position");
                    float x = headLocation.pose.position.x;
                    float y = headLocation.pose.position.y;
                    float z = headLocation.pose.position.z;
                    *X = x;
                    *Y = y;
                    *Z = z;
                }
                else
                {
                    XR_MESSAGE("Failed To Get Head Position (2)");
                    *X = 0; *Y = 0; *Z = 0; // Default or error values
                }
            }
            else
            {
                XR_MESSAGE("Failed To Get Head Position (1)");
                *X = 0; *Y = 0; *Z = 0; // Default or error values
            }
        }
        void SetGameEnginesCameraAngle(float *QuatW, float *QuatX, float *QuatY, float *QuatZ, XrTime predictedTime)
        {   
            *QuatW = 0;
            *QuatX = 0;
            *QuatY = 0;
            *QuatZ = 0;

            XrSpaceLocation headLocation{XR_TYPE_SPACE_LOCATION};

            XrResult result = xrLocateSpace(m_stageSpace, m_viewSpace, predictedTime, &headLocation);

            if (XR_SUCCEEDED(result))
            {
                if  ((headLocation.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT)    != 0 && 
                     (headLocation.locationFlags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT) != 0 )
                {
                    XrQuaternionf q = headLocation.pose.orientation;
                    float qx = q.x;
                    float qy = q.y;
                    float qz = q.z;
                    float qw = q.w;
                    
                    *QuatW = qw;
                    *QuatX = qx;
                    *QuatY = qy;
                    *QuatZ = qz;

                    XR_MESSAGE("Success: Head Quaternion");
                }
                else
                {
                    XR_MESSAGE("Failed To Get Head Quaternion (2)");
                    *QuatW = 0; *QuatX = 0; *QuatY = 0; *QuatZ = 0; // Default or error values
                }
            }
            else
            {
                XR_MESSAGE("Failed To Get Head Quaternion (1)");
                *QuatW = 0; *QuatX = 0; *QuatY = 0; *QuatZ = 0; // Default or error values
            }
        }

Any advice is appreciated. Thanks

  1. could be an issue with the projection matrices or the poses derived from xrLocateViews. Most VR headsets don’t have symmetric display dimensions, so make sure the engine is set up to handle off-center projection matrices.

  2. The calls to xrLocateSpace in your code has the spaces reversed. They should locate the VIEW space relative to the STAGE space. Note that his only gives the center of the head pose, and you need to compose it with the eye pose from xrLocateViews to get the pose to render from (or just call xrLocateViews with the STAGE space to get the complete pose directly).

XR_REFERENCE_SPACE_TYPE_LOCAL is typically used for seated games where you don’t need to know where the floor is, like games where you sit in a cockpit.

Thanks for the reply. It helped me get a bit further in my project.

Also, I figured out that the game engine (using android’s OpenGLES) is flipping the image vertically. And
this may be causing the left and right eye image to be reversed. Temporarily switching the projection matrix on the wrong eyes, the double image goes away. I think I just need to look through the engine’s source code and see how I can flip the image.