Question about RGB and true (displayed) colors

Hello,

I have a problem with displayed RGB colors.

When I’m creating render target for OpenXR:
swapchain_info.format = DXGI_FORMAT_R8G8B8A8_UNORM_SRGB
(if I try to create it with DXGI_FORMAT_R8G8B8A8_UNORM, it will fail and run program in windowed mode instead)

When I’m creating render target for windowed mode:
FrameTextureDescriptor.Format = DXGI_FORMAT_R8G8B8A8_UNORM
(with DXGI_FORMAT_R8G8B8A8_UNORM_SRGB output colors are the same)

The problem:

displayed colors in windowed mode are miles away from colors inside headset.
I know that Virtual Desktop Streamer changes colors a bit (I turned vibrant colors on and off, but that is not the core of the problem).

In headset colors are PINKISH. Gray color is not gray, it has a pink hue (while gray on desktop is gray). Inside the headset lenses everything is so obnoxiously pink, that I had to create a final color correction in pixel shader to fix this (it has essentially a blue bias).

However, if i make a VIDEO inside a headset:

  • with pixel shader color correction, video will be masively blue, while displayed colors in the lenses will be nice
  • if I turn final PS color correction off, then video from headset will be fine, but the image in lenses will be pinkish

Video from headset without color correction is here: https://www.youtube.com/shorts/m2B_tD4wSYw

Video from headset with color correction is here:https://www.youtube.com/shorts/hPjuSrZxPQ4

Image through lenses without color correction is here (taken with phone, that’s the best I can do).

Notice the ICP panel below head up display. The ICP keys (0-9) should be gray, and other gray structures in the cockpit should be gray, but are pink.

Through the lenses, with no color correction, gray is pink.
In video with no color correction, gray is gray.
Through lenses with color correction, gray and other colors are “relatively” fixed, but colors thne broken in video.

I haven’t found any workaround for this except to make color correction for headset.

When I was investigating the problem, I created a surface in front of my eyes that I could change its color by setting RGB 0…1 with keys. In that way I was looking for known RGB combination and observing what colors I actually see. Then I went and made color correction curves to fix colors according to their RGB input - to fix gray to be actually gray, magenta to be magenta, yellow to be yellow etc. Black and white are no problem because they are actually saturated, so are pure R or G or B not problematic as they are anyway saturated. Problematic are combinations of RGB when values are between 0 and 1.

(with no color correction:)
What I noticed was:

RED color: 0 = black, 0.65 = maximum
if I increased red color above 0.65, color inside lenses did not change any more, it was at maximum, no difference between colors R=0.65 and R=1.00

GREEN color: 0 = black, 0.90 = maximum
if I increased green color above 0.90, color inside lenses did not change any more, it was at maximum

BLUE color, the most broken color in lenses
0 = black, 0.90 = maximum (if I remember correctly)
but blue is underrepresented at values < 1

What that means is, if I input RGB = 0.5, 0.5, 0.5
red will be almost at maximum intensity (0.65 = max), green will be approximately at half, and blue will be less than half visible intensity, making such gray color look pink, because there is too much red and too little blue.

The color correction curves that I came up with are on the image above.

X axis is input value.
Y axis is output value that is needed to achieve a certain color.

It is not a perfect correction. I corrected yellow, magenta, cyan, some gray colors etc… but now in some areas gray is a tiny bit too much blue tint.

As you can see blue has “gamma” curve above neutral “1” line, because I needed more blue value than red or green, to achieve gray output.

Now, if anyone knows where lies the problem.
Did I do something wrong to get wrong colors in headset lenses?
Is there some setting that I do not know of, that is causing this?
Virtual Desktop Streamer has color correction, but it is not the cause of this (even if I turn it off, problem remains).
And the difference is already in headset, video recorded in the headset has different colors than lenses.

Is there any other solution to this other than manual color correction?

Thanks for any answers.

Do other OpenXR applications experience the same issue? Checking it should be a quick way to determine whether the issue is with the application or runtime. I think most applications submit with DXGI_FORMAT_R8G8B8A8_UNORM_SRGB by default.

Hey,

I just compiled the Microsoft OpenXR-MixedReality project from Github, BasicXrApp.sln

Problem persists.

Below is my modification of that project. What has been modified:

  • pixel shader was changed so that it neglects vertex buffer color and takes color from constant buffer (this color is dynamically set)
  • I changed how controller actions work
  • right B: cycle between R → G → B → R … components
  • right A: cycle between R → B → G → R … components (reverse order)
  • right thumbstick up/dn: increase decrease selected RGB component
  • left X: truncate R/G/B component to multiple of 0.1 or decrease by 0.1
  • left Y: roundup R/G/B component to multiple of 0.1 or increase by 0.1
  • app opens a console window and displays current RGB values there

Cubes start with a color of (0.5, 0.5, 0.5) which should be gray. In my lenses that is pink.
If I increase RGB colors towards 1, cube becomes more and more white, and error is not so obvious. Gray colors below 0.5 are increasingly pink in hue.

I also tried to make cubes black and then I increased Red component to the point where it stopped visually increasing. The value (displayed on console window) was around 0.5 (above which increasing R was not producing a visible change in lenses)

I tried to pinpoint the visual maximum for Green (came around 0.95) and Blue (also around 0.9) although blue is visibly less intense than red or green, thus gray colors are pink.

So, in this app I did not touch anything that is connected to creation of swapchain / render buffer, I left it as it is on github, and I still see gray colors as pink (I did not dive so deeply into other color combinations as I did in my own project).
I looked at this app just to see (as you suggested) if problem exists in other XR apps or not.

As I said above, I have no idea where the problem lies for this, because the headset (as I understand) receives the same image (stream) from computer graphics card, be it that you watch it through lenses or make a video in the headset with that data. The video is colored correctly, while image in lenses is not.

Maybe someone know why this happens and how to fix it?

Modified code:
https drive google com / file / d / 1j7nHOVkle5aD8fsxN43SubwyZjrNMizh

Sounds like it’s not an issue with the application then.

The chain for presenting an image would roughly be:
Application → [API Layers] → Runtime → Encoder → Headset → Decoder → Compositor

If the headset system UI is displayed correctly, the problem would either lie with any API layers installed, or one of the steps controlled by the streaming software. I suggest testing with alternative streaming software/runtimes.

I ran your app with Virtual Desktop + SteamVR OpenXR and also VDXR, and I see no problem. At 0.5/0.5/0.5, the cubes do not look pink at all in the headset for me.

Suggest trying with SteamLink or Quest Link as Rectus mentioned. If the problem is specific to Virtual Desktop, suggest heading to their Discord and describe your issue there. They’ll likely look at your settings and see if anything looks out of the ordinary.

PS: BTW it looks like you are working on something similar to one of my first API layers

A feature I later integrated into my OpenXR Toolkit but eventually removed due to lack of time.

Hope you will come up with something great!

It is a part of a larger project that I’m working on, converting that one to VR and adding new functionalities. What was meant to be a simple new graphical engine now overgrew the old goals and is becoming a more general-use library. What can be seen on pictures/video is actually a test-bed for the library, just for testing all of the functionality. The underlying project is currently on pause, until I finish this graphics class, then I will marry the two, hopefully soon, and continue on the main project. It is just something I do for my own amusement and for the love of it.

Thanks for the kind words :slight_smile: