[Custom Unity Plugin] World Locked Composition Layer

[OwlchemyDawson]

I am looking for help on better understanding Spaces and how to properly use them to place a static, non-moving quad composition layer in the world.

I am working within a C++ Unity rendering plug-in that I am developing to support composition layers. I am able to render custom layers at the correct world space location using my plug-in; however, I have found that the placed composition layer I am adding in the world is moving with the player when the player root is moved (i.e. the player teleports in the game). I am seeing this behavior in Unity Editor version 2021.3.13f1 (and 2021.3.24f1) using a Meta Quest 2 and OpenXR package version 1.7.0 with no other device specific XR packages (i.e. no Oculus, SteamVR, etc).

When creating a composition layer to be rendered, I created a XrSpace using XR_REFERENCE_SPACE_TYPE_STAGE as the reference space (after ensuring STAGE is available) with an identity pose. The intent here is to place a composition layer in a room-scale player area relative to the floor. The simplified version of the reference space creation logic I am using is as follows and the xrCreateReferenceSpace returns successfully:

XrReferenceSpaceCreateInfo spaceCreateInfo = { XR_TYPE_REFERENCE_SPACE_CREATE_INFO };
spaceCreateInfo.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_STAGE;
spaceCreateInfo.poseInReferenceSpace.orientation.w = 1.0f;
				
xrResult = OpenXRUtility::xrCreateReferenceSpace(xrSession, &spaceCreateInfo, &referenceSpace);

My understanding of STAGE space is that it is a walkable room-scale rectangle with an origin at the rectangle center on the floor and it should not move. As mentioned above, I have found that after placing my composition layer in the world it moves with the player whenever the player’s root is moved (i.e. they teleport in the game).

Am I misunderstanding something about STAGE space and how to render a world locked composition layer? Any pointers on a proper way to implement this functionality?

[ryliepavlik]

That does sound odd. How is the “player’s root being moved”? (How does teleport work?)

If the layer’s space is fixed, perhaps the views are being rendered relative to a space that is not fixed, which is causing your change? (assuming you’re not doing something like a quad layer, which should be immune to this kind of thing) So this might only happen for projection layers, in which case you typically only want one of those rendered for the view…

I tend to go back to this “Flying, Grabbing, and Scaling in Virtual Worlds” paper for stuff like this: CiteSeerX OpenXR doesn’t separately expose a tracker base relative to the room, but the diagram still fits pretty close:

Room in the diagram would be like stage space, while “world” is your virtual world. Head is like reference space view, left and right eye are found with xrLocateViews. I would logically expect that “teleport” would change the link between (virtual) world and “room” (stage). Maybe this helps explain the issue or even figure it out?

I suppose the real question would be what kind of layer you’re submitting and how you’re setting up that submission, because if you submit a non-projection layer in the space created as described, it will be (real) world locked (moving in the virtual world on teleport).

[OwlchemyDawson]

With regard to teleporting, it works by updating the Unity GameObject Transform that is the parent of the VR camera.

I am using a quad composition layer that I inject at the end of the XrFrameEndInfo layers data during xrEndFrame; the layer is added after the project layer. After creating a swapchain for the layer, retrieving the native texture pointer, and creating a reference space as described above then I create a XrCompositionLayerQuad as follows:

auto newQuadLayer = new XrCompositionLayerQuad();
newQuadLayer->type = XrStructureType::XR_TYPE_COMPOSITION_LAYER_QUAD;
newQuadLayer->layerFlags = XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT;
newQuadLayer->space = m_LayerSpace;
newQuadLayer->eyeVisibility = XrEyeVisibility::XR_EYE_VISIBILITY_BOTH;
newQuadLayer->subImage = XrSwapchainSubImage{ layerSwapchain, { {0, 0}, {(int)texture.width, (int)texture.height} }, 0 };
newQuadLayer->pose = XrPosef{ {rotation.x, rotation.y, rotation.z, rotation.w}, {position.x, position.y, position.z} };
newQuadLayer->size = { size.x, size.y };

If I am understanding correctly, it sounds like I need to create a new reference space with an appropriate pose based on the player root’s new location every time the player teleports and update all layers to use that XrSpace instead. Would my understanding be correct here?

[OwlchemyDawson]

I implemented reference space recreation on player teleport using the inverse of the VR camera parent’s Transform data and got world locked composition layers working properly. Thanks for all of the help, @ryliepavlik.