I was just wondering, in the open-source world I don’t think there’s yet any good code for tracking HMDs with cameras. You need special high-speed cameras. So I don’t know if that’s a good explanation for why. But OpenXR is an API for doing VR stuff, so I wonder if anywhere inside it there is general code for talking to cameras in an abstract way?
OpenXR is an API, not an implementation, so there’s no “code” in OpenXR aside from utility things like loaders and validation layers. Some implementations of OpenXR may have such code.
There is a bunch of camera-based tracking code, but it’s mostly included in other runtimes. e.g. there’s actually pretty flexible tracking code in OSVR, though the default usage requires a camera synchronized with patterned blinking of lights. I split that code out of OSVR for easier re-use here: https://github.com/rpavlik/UVBI-and-KalmanFramework-Standalone (there’s also a really nice little Kalman Filter framework in there called FlexKalman, it’s header-only and very fast and flexible). The project I currently work on, Monado, is an OpenXR runtime and it does contain some code for tracking the PSVR headset’s LEDs, though it’s work-in-progress https://monado.freedesktop.org/ I think there’s also some other PSVR tracking projects out there that get used with Trinus or iVRy?
I will be taking a look at your code. I have some history with PSVR and actually by random chance I struck up a conversation with the iVRy owner the other day that reminded me of the dearth of public domain camera tracking solutions (https://www.reddit.com/r/PSVRHack/comments/hxemqc/any_active_psmoveservice_devs/)
But I ask if OpenXR provides APIs for camera devices. Of course, if OpenXR weighs in on this each device must supply implementations. Like if OpenXR talks to sets and other VR controllers, it seems like cameras are a natural kind of device it could provide some structure to just like controllers and other kinds of trackers. There is OpenCV (OpenVX?) but these are maybe too broad for the application and it would be nice to not have to deal with a second API if already using OpenXR.
Like ideally OpenXR could abstract using cameras to the point software can easily accommodate user’s needs and users could just decide if their cameras are right for their needs or not. Then this would lift a lot off developers like OpenGL (for instance) saves us developers from the internal details of graphics hardware.
FYI: I noticed on your project’s page PSVR says “improved distortion correction is WIP” so I can offer to help you there since I believe I have perfect correction in my software. And I think I have very good (almost no drift, better than Sony’s cinema mode) code for integrating the internal sensors too. The iVRy person said they’ve worked with you, so maybe they can help to foster a group between us since they’re trying to solve the camera problem right now (they want to do it in the next 3 months, but my schedule isn’t open until next year.)
P.S. I think it would be really good if a Monado project could provide OpenXR for multiple devices but also expose a low-level back-end that can be used to go around OpenXR. I think this would be a very beneficial project. Just a way to not limit progress to OpenXR.
We’d certainly be happy to have assistance with the distortion! (Do I know of your project?) I’m not familiar with the identity of the person behind iVRy, perhaps we both participated on some GitHub issue unknowingly? I’d suggest that both you and the iVRy person come to the Monado Discord channel so we can chat and see if we can put together a better solution. https://discord.gg/8RkJgRJ
They said the following in one of the Reddit replies that must have got filed under a “Continue this thread->” link. I say they because I don’t think I caught their name or gender. Maybe I should’ve asked.
“Thanks for that lead. I’ll get in touch with Ryan. I worked with him on a SteamVR driver for OSVR dual display headsets.”
(I think I’m going to carry on the PSVRTracker project next year some time. I do game stuff, and using a PSVR was a simple/inexpensive way to clone my regular 1920x1080 display to develop some features last year. If I didn’t say so I thought about developing an OpenXR wrapper for PSVR so I’m curious to see both how that effort is going with your project–what that looks like–and also if you have any useful camera code.)
I joined your server, but I don’t use Discord a lot, so hopefully “muting” it is enough that it doesn’t become a nuisance. I will stumble into the chat room as soon as I find time to study your code. I’d invite you to direct-message me but I don’t know if this forum has a system. I could give you code links. I don’t mind posting them publicly but it’s veering off topic and the code is part of game logic.
Edited: If you can find my account (m.) in your Discord users, feel free to chat me up 
This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.