I’m wondering if it’s possible in principle to build an application that behaves like a runtime, inserting itself between the active runtime and subsequent application launches?
This would be an application that connects to an active OpenXR runtime on the system using the standard runtime discovery process, and then proceeds to launch other OpenXR applications while overriding runtime discovery to point to itself, intercepting calls and passing them through to the underlying runtime as necessary.
The idea here is to build an application that provides shell functionality - an opinionated user interface layer for managing multiple applications, composition, focus and input routing. Is it possible to build this kind of OpenXR compatible shell without having to build an entirely new runtime from scratch?
What you’ve basically described is a runtime, or at least, a dashboard/system ui app. That’s one of the reasons there isn’t device plugins for headsets yet - folks who make headsets want to control and provide such an “opinionated UI layer” in their system UI. That experience is very important to the overall headset experience.
If you’d like to experiment with building such a UI, you might look into Monado, which is an open-source OpenXR runtime I (along with others) work on. We have a lot of features, but pretty minimal system/dashboard UI, and would appreciate contributors there: we’ve mainly been focused on the system/framework level functionality. Such a dashboard app would need some special access beyond what the unextended OpenXR API provides, and since the only public API for Monado is OpenXR we have planned basically on just creating a vendor extension for whatever those system UI capabilities are, with only one application allowed to use that extension at a time, and possibly other user-preference-related protections. See https://monado.freedesktop.org/ to get involved!
I checked out Monado and it looks like a really cool project. My biggest concern with it is the lack of support for more commonly used headsets and software platforms. I understand this stuff is pretty difficult work and that it will take time. I’m glad that these problems are being actively worked on.
From what you describe, it sounds like any dashboard/system UI needs to be closely tied to the runtime itself, rather than being “just another” OpenXR app. You mention that a dashboard app would need special access - can you elaborate on this? What specifically would be possible using the unextended OpenXR API without special access?
I’m not used to working at this level so I might have some deep misunderstanding of what’s actually possible here. I’m working on getting a prototype/proof-of-concept working but it would be good to know if there’s something that makes this totally impossible.
All headset drivers in Monado mainline, aside from the North Star and the OSVR HDK, are based on reverse-engineered protocols from the hardware, and positional tracking of any sort (but especially SLAM camera-based tracking such as on winmr headsets) requires a lot of software computation in addition to interpreting the data being received from the headset, which is why drivers there are scarce. Though, much of this could be tested/prototyped using the simulated headset which would just display the image in a window on your desktop.
RE special features: Well, at a minimum you’d need to be able to overlay other content, and also to switch which application is considered “active”. There’s a very experimental extension for overlays in OpenXR (that Monado has implemented), and Monado has an internal API to set the active application. You would also want to be the only one receiving events from the “system” button on the controllers, and to receive that even when you’re not the active application: both of which are outside of the standard spec. (the system buttons are listed as “may not be available for apps” so it’s fine to deprive other apps of them, but you’d want the dashboard to exclusively own them.
There’s nothing inherent that requires it to be tied to a given runtime: anybody could implement extensions like this and publish them. However, since the dashboard experience is such an important part of the overall user experience of a headset, vendors are understandably often quite protective of their ownership of this part of the system. I don’t know if the APIs to make a dashboard in SteamVR are public, but that’s the only runtime I know of where there’s even an option to choose a dashboard.