Quest 2 controller rendering

How to get the models for rendering the controllers for these headsets? Before I used a mixed reality headset which supported this extension: XR_MSFT_controller_model

My headset supports the extension: XR_META_detached_controllers
but I can’t find any information on this at all, it doesn’t seem to be listed in the specs

The detached controller one is something else.

For controller models, I believe you need to use that one instead: XR_FB_render_model

XR_FB_render_model is not listed as an extension I can use. This is my extension list

  • XR_EXT_hand_tracking
  • XR_KHR_D3D11_enable
  • XR_KHR_D3D12_enable
  • XR_KHR_opengl_enable
  • XR_KHR_vulkan_enable
  • XR_KHR_vulkan_enable2
  • XR_KHR_composition_layer_depth
  • XR_KHR_win32_convert_performance_counter_time
  • XR_KHR_convert_timespec_time
  • XR_KHR_composition_layer_cube
  • XR_KHR_composition_layer_cylinder
  • XR_KHR_composition_layer_equirect
  • XR_KHR_visibility_mask
  • XR_KHR_composition_layer_color_scale_bias
  • XR_EXT_win32_appcontainer_compatible
  • XR_EXT_debug_utils
  • XR_OCULUS_recenter_event
  • XR_OCULUS_audio_device_guid
  • XR_FB_color_space
  • XR_FB_display_refresh_rate
  • XR_META_performance_metrics
  • XR_META_headset_id
  • XR_OCULUS_ovrsession_handle
  • XR_EXT_local_floor
  • XR_EXT_active_action_set_priority
  • XR_FB_haptic_amplitude_envelope
  • XR_FB_haptic_pcm
  • XR_FB_touch_controller_pro
  • XR_META_touch_controller_plus
  • XR_FB_touch_controller_proximity
  • XR_OCULUS_external_camera
  • XR_META_detached_controllers

Oh you are on PC. Then I believe there is no option for you. WMR was the only PC platform with controller model support in OpenXR AFAICT.

You may be able to use Oculus Developer mode to get access to XR_FB_render_model, but it’s going to be an issue for your users (they will also need to use Developer mode, which is not straightforward).

lol what, seriously?
How are other people solving this problem? Just using 3rd party models or something?

yeah it’s a harder problem to provide this in a forwad-compatible manner than you might expect (that is to say, providing it in a way that avoids assumptions.)

But basically, yes, content authors bring their own models at this time. OpenXR-Guide/chapters/ at main · KhronosGroup/OpenXR-Guide · GitHub

Has there been any progress in developing a multi-vendor extension for rendermodels? Most runtimes have had support in their proprietary APIs from before OpenXR was a thing, and it seems kinda against the design philosophy to implement code that switches of a limited set of input devices.

This seems like an absolutely insane oversight in the spec, models should not be optional

Well I tried loading the quest2 models from steamvr, the obj models. And basically it works but the orientation and position are off somehow. I know openxr has 2 positions, one is the ‘grip’ position, and the other is for a laser pointer.

Does anyone know the correct translation to draw these models? Or have any other suggestions for loading controller models?

I don’t think its documented anywhere. The SteamVR models are aligned to the poses the individual OpenVR drivers output, which seem to be different from the OpenXR poses in most cases. The only way I can think of to get the translation would be to write an application that runs both OpenXR and OpenVR in background mode, retrieves the poses for both, and calculates the difference.

Yes, there is progress. It’s a tricky problem to solve in a way that satisfies all our design requirements and priorities, but it’s been reported that we have work in progress close to completion.

However, switching model based on current interaction profile is not against the design intent, it’s the intended use of that capability. See the OpenXR-Guide link above.

1 Like

Thanks Rylie! It’s great to hear that it’s progressing.

I managed to get controller rendering working using the steamvr models but it was a kludge. I guessed the angle I think correctly but the position is not 100% true according to what it should be, but it’s close enough. But the whole process has been … just an uphill battle. Basic controller rendering should have existed in version 1.0 of the spec, and been a core part of the spec. Microsoft did it … with their own extension. The whole point of openxr is to avoid fragmentation? My experience it has been worse than ever with this API. Worse as well for me because various runtimes (eg microsoft) didn’t support opengl, so I had to write a ridiculous d3d->opengl shared window solution. SteamVR itself works something like this, but randomly supporting or not supporting various apis … just leads to more fragmentation.