So I am in the process of transitioning VR support in our application from native libs to OpenXR. We use eye tracking for analytics and research, so trying to maximize accuracy is important.
I have some questions about XR_EXT_eye_gaze_interaction. I find the extension pretty lacking, feature wise, compared to the native libs, such as SRanipal for the Vive. But I’d like to make sure this isn’t an oversight on my part, maybe the data is available in another way I haven’t seen. I’m testing on Vive Pro Eye and Vive Focus 3 with eyetracking addon.
When calling xrGetSystemProperties with the XrSystemEyeGazeInteractionPropertiesEXT extension property, to check for eyetracking availability, the supportsEyeGazeInteraction flag is always true on the HTC Focus 3, even when the eyetracking addon is physically disconnected from the headset. This looks like a bug.
Is there any way to query the refresh rate of the eyetracker ? This is separate from the HMD screen refresh (and usually much higher). We query the eye tracking poses in a separate thread at much higher frequency than the screen refresh. Querying at (or near) the native refresh rate of the eye tracker is important to avoid interpolation.
Is there a way to detect eye blinking ?
Is there a way to force recalibration of the eye tracker from the OpenXR application ? Our use case involves high volume use, ie. the tracker will be used on multiple different people during a research session, often people with very different physical sizes, ages, etc. Recalibration between participants is very important for accuracy. Previously we would automatically trigger recalibration for every new person using the native SDK. Is there any way to do this from OpenXR ?
I agree, this looks like a bug. TBH, I thought they had fixed that a while ago. But yes, follow up with HTC Vive on this one.
Not directly, but I think this is what the eye gaze sample time is for, so you can re-request the eye gaze data for the exact sample time. Also the spec tries to say there won’t be interpolation.
It looks like that is meant to be inferred from whether the gaze is position tracked, I think.
Not from this extension: that is explicitly left to the system. This sounds like an excellent candidate for an additional extension for use cases like yours, where the app has more information or tighter requirements about eye tracking than the runtime might. I recommend talking to your vendor and suggesting they propose one. Seems like it could probably be pretty simple, just a single function effectively wrapping that native API you refer to.
It also seems to be affected by HMD sleep. When creating the instance while the HMD is asleep, the flag will not be set on wakeup even if the eye tracker addon is connected. The instance will need to be recreated for the flag to be updated.
I found this part of the spec to be confusing. While it sounds like current runtimes would not predict / interpolate (and the spec requires them to clamp time to the actual sample time, which makes sense), the specs seems to allow it, basically saying that its up to the runtime to decide:
The runtime must set time in the XrEyeGazeSampleTimeEXT structure to the clamped, predicted or interpolated time. [...]
The time field may be in the future if a runtime can predict gaze poses
Right, that would make sense. Unfortunately that part is bugged in my runtime, the pose always comes back as tracked, even when its clearly not.
What would be the best way to approach this ? Is there a procedure one can follow if one is not directly a Khronos member ?
I’ll join this thread because it is the closes I could find on the Internet.
We are faced with difficulties similar (but not quite) to the one @AlexDev has described.
Similar to you, we are trying to use the XR_EXT_eye_gaze_interaction in conjunction with a Vive Focus 3 with eyetracking addon.
While he describes that xrGetSystemProperties with the extension property XrSystemEyeGazeInteractionPropertiesEXT always returns “true”, we are faced with the opposite: after querying the SystemProperites, supportsEyeGazeInteraction is always set to false.
@AlexDev Which software toolchain are you using to get the eyetracking to work? Would really appreciate a feedback since your thread is the closest we have found to our situation.