Quest 2 controller rendering

How to get the models for rendering the controllers for these headsets? Before I used a mixed reality headset which supported this extension: XR_MSFT_controller_model

My headset supports the extension: XR_META_detached_controllers
but I can’t find any information on this at all, it doesn’t seem to be listed in the specs

The detached controller one is something else.

For controller models, I believe you need to use that one instead: XR_FB_render_model

XR_FB_render_model is not listed as an extension I can use. This is my extension list

  • XR_EXT_hand_tracking
  • XR_KHR_D3D11_enable
  • XR_KHR_D3D12_enable
  • XR_KHR_opengl_enable
  • XR_KHR_vulkan_enable
  • XR_KHR_vulkan_enable2
  • XR_KHR_composition_layer_depth
  • XR_KHR_win32_convert_performance_counter_time
  • XR_KHR_convert_timespec_time
  • XR_KHR_composition_layer_cube
  • XR_KHR_composition_layer_cylinder
  • XR_KHR_composition_layer_equirect
  • XR_KHR_visibility_mask
  • XR_KHR_composition_layer_color_scale_bias
  • XR_EXT_win32_appcontainer_compatible
  • XR_EXT_debug_utils
  • XR_OCULUS_recenter_event
  • XR_OCULUS_audio_device_guid
  • XR_FB_color_space
  • XR_FB_display_refresh_rate
  • XR_META_performance_metrics
  • XR_META_headset_id
  • XR_OCULUS_ovrsession_handle
  • XR_EXT_local_floor
  • XR_EXT_active_action_set_priority
  • XR_FB_haptic_amplitude_envelope
  • XR_FB_haptic_pcm
  • XR_FB_touch_controller_pro
  • XR_META_touch_controller_plus
  • XR_FB_touch_controller_proximity
  • XR_OCULUS_external_camera
  • XR_META_detached_controllers

Oh you are on PC. Then I believe there is no option for you. WMR was the only PC platform with controller model support in OpenXR AFAICT.

You may be able to use Oculus Developer mode to get access to XR_FB_render_model, but it’s going to be an issue for your users (they will also need to use Developer mode, which is not straightforward).

https://github.khronos.org/OpenXR-Inventory/extension_support.html#XR_FB_render_model

lol what, seriously?
How are other people solving this problem? Just using 3rd party models or something?

yeah it’s a harder problem to provide this in a forwad-compatible manner than you might expect (that is to say, providing it in a way that avoids assumptions.)

But basically, yes, content authors bring their own models at this time. OpenXR-Guide/chapters/goals_design_philosophy.md at main · KhronosGroup/OpenXR-Guide · GitHub

Has there been any progress in developing a multi-vendor extension for rendermodels? Most runtimes have had support in their proprietary APIs from before OpenXR was a thing, and it seems kinda against the design philosophy to implement code that switches of a limited set of input devices.

This seems like an absolutely insane oversight in the spec, models should not be optional

Well I tried loading the quest2 models from steamvr, the obj models. And basically it works but the orientation and position are off somehow. I know openxr has 2 positions, one is the ‘grip’ position, and the other is for a laser pointer.

Does anyone know the correct translation to draw these models? Or have any other suggestions for loading controller models?

I don’t think its documented anywhere. The SteamVR models are aligned to the poses the individual OpenVR drivers output, which seem to be different from the OpenXR poses in most cases. The only way I can think of to get the translation would be to write an application that runs both OpenXR and OpenVR in background mode, retrieves the poses for both, and calculates the difference.