According to the OpenXR spec, action inputs are received only when the session state is XR_SESSION_STATE_FOCUSED, and focused state is active only when the headset is on. This means that I can get controller pose state while the headset is off but there is no way to get controller button state.
Two questions:
Is there a good reason for this design decision? Why shouldn’t it be possible to continue to use controller buttons when the headset is not on?
Is there any way to work around this? I could not find any way to force the session state to change or to access the button state without having active action inputs.
When you lose FOCUSED, it probably means that a dashboard is pulled up or some other case where your application is not active. Some pose data may flow (to allow head motion to work in the environment, so your world doesn’t freeze when you pull up a dashboard or system menu), but not all input data will flow (so your app can’t sniff e.g. a credit card number being entered in a store interface).
Can you point out where you saw that you will get controller pose when not focused? I want to make sure it’s clear there that you may get it, but (e.g. in the on-screen-keyboard credit card scenario) you aren’t guaranteed it. I think (off the top of my head) you might be guaranteed head/view pose maybe?
If the headset is turned off, I would suspect you would not get any data, you’d probably get a session exit event flow. If the headset is not worn, your session state will also change, but I think that it will go further than just “not focused”. If you want the input data without rendering to the headset, let your runtime vendors know they should work with others in the group to firm up a multi-vendor “headless” extension (the Monado runtime I work on has such an extension, though work on a better extension is somewhat constrained by fitting in to the session lifecycle you’ve mentioned…)
To be clear, I am talking about the headset still turned on but removed from the user’s head, and a sensor is in the headset sends a signal that it is no longer being worn. When this happens (at least in my experience), the state changes from FOCUSED to VISIBLE and button input no longer works. Pose input continues to work, although I’m not sure the spec mandates that.
I do not want to go to a full headless session, as I want the headset to work when put on. I just want input to work when it is taken off.
The spec seems very ambiguous on this, so there is likely a lot of leeway on how runtimes choose to implement it.
I’m guessing your issue comes from using SteamVR. Since the last few betas they changed it to very aggressively changing the session state when the headset is taken off, along with only enabling the input profiles when the headset is worn. Since it’s not possible to turn off the behavior, it makes development difficult with forcing you to take the headset on and off constantly. The interaction profile behavior is also not clearly documented, leading to it being interpreted as a bug.
OK, that’s what I figured you were talking about, but I put the stuff about headset-entirely-off in there just in case I was wrong.
If you do xrSyncActions when your session is not focused (so, headset doffed, for example), the spec says: " If session is not focused, the runtime must return XR_SESSION_NOT_FOCUSED, and all action states in the session must be inactive."
It is intentionally left to the runtime how they want to drive the focused/visible parts of the state: we didn’t want to say “if you have a user sensor” and thus imply you can’t use e.g. IMU data to detect the headset was set down, etc.
This is a problem hitting me too. I’m right now trying to port OpenVR over to OpenXR due to potential problems with SteamVR over OpenVR on Windows.
The situation I have is this. I’m doing (besides other things) a motion capture tool using VR. One mode of operation is with HMD put on but a second, very important mode is using only a tracker head-band with the HMD resting on a table or the floor. Using OpenVR with SteamVR this works without a problem. But with OpenXR I run into a problem since tracking stops if the HMD is taken off.
I see “XR_MND_headless” not being reported by my SteamVR OpenXR implementation but headless works using OpenVR (and SteamVR OpenVR is pretty much just bridging to SteamVR OpenXR, IMAO). Is there no way OpenXR can be made to not stop tracking?
The spec doesn’t seem to allow for headless operation without that extension. Valve doesn’t seem to have expressed any intent to support headless operation in OpenXR though.
I was hoping for Valve to at least add a developer toggle for the headset detection, but instead they seem to be gearing up to push the more aggressive behavior from the beta to the stable SteamVR branch.
Maybe there is a more “make shift” solution for this problem? The headset has to detect somehow if it is put on or taken off. Maybe this sensor can be tricked? Not the nicest solution on earth but if OpenXR fails to provide such an important use case maybe something like this might help?
That said… I’m still puzzled why OpenVR continues tracking if the headset is taken off (without bringing up the SteamVR menu) but OpenXR stops tracking pretty much the instant you take it off (well… 1 second delay or so in my tests).
I’ve got a HTC. You happen to know if I can trick the sensors there too? This way users could simply block the sensor while doing mocap and unblock it again when done.
Indeed Monado is currently the only OpenXR runtime where the headless extension is available. But it shouldn’t be an absolute requirement for this to work.
There are two behaviors relating to “engagement sensors” described in the spec:
If the system supports a user engagement sensor and runtime is in XR_SESSION_STATE_IDLE state, the runtime should not transition to the XR_SESSION_STATE_READY state until the user starts engaging with the device.
and
If the system supports a user engagement sensor and the session is running, the runtime should transition to the XR_SESSION_STATE_STOPPING state when the user stops engaging with the device.
As this says should and not must, runtimes are free to make other behaviors available and still be compliant to OpenXR.
The two settings I see in SteamVR are
In the Video settings “Pause VR when headset is idle”.
This also does not affect the session states controlled by the proximity sensor.
Your best bet is to ask to have a similar setting added to SteamVR that controls proximity sensor behavior. Until then all you can really do is physically cover the proximity sensor.
With OpenVR I did not need this to be done. With OpenXR it seems to be necessary. I have now a business card leaning against the sensor. Leaves no traces and is easy to remove/put in place.
(the Monado runtime I work on has such an extension, though work on a better extension is somewhat constrained by fitting in to the session lifecycle you’ve mentioned…)
