Image orientation for cubemaps (actually a very old topic)

I am trying to understand the reasoning behind the image orientation for cubemap images displayed on a skybox. I found nothing helpful, mostly a lot of misinformation, in an internet-wide search. I found two very old posts in this forum which are on topic:

but they aren’t helpful as people keep talking about flipping images without being specific about the orientation. There are links in both to a .ppt from @cass said to explain the reasoning but the link returns error 404. There is also a link to an old opengl.org forum post, also seemingly by @cass, which likewise returns 404.

So could someone please provide exact information as to the requirements and reasoning behind it.

In the following descriptions my starting point is photographs of the scene matching the ground truth.

What I have found is that, on OpenGL, images with the default bottom-up orientation have to be flipped vertically and all images have to be flipped horizontally - something most people never seem to have noticed. Actually no flipping is necessary, you can just apply a scale of -1, -1, 1 to the uvw coordinates being passed in to the cube sampler.

On Vulkan images with a top-down orientation need to be flipped or a -1 scale applied to the y coord. If you try with bottom-up oriented images you don’t need y*-1 for the images to have correct vertical orientation but the sky and ground will be swapped. You have to swap the posy and negy images in the cubemap for this to work. A -1 scale needs to be applied to the x coord, as with OpenGL, regardless of the image’s y orientation.

All this is in line with the face selection table and equations in the OpenGL & Vulkan specs. But I am completely failing to understand why it was designed this way.

All the OpenGL samples I have looked at in the wild load the cubemap from individual .jpg or .png files. Since these formats have a top down orientation the samples just work. Except for the left-right issue.

I have found Vulkan samples using ktx files with posy and negy swapped whose images have a bottom up orientation (mislabelled as having top down) so they seem to just work without any scaling of the uvw coords. Except for the left-right issue.

None of the samples or tutorials I’ve found talk about the need for images to be oriented differently than when used in an ordinary 2D texture.

What I am really seeking is a way to load the exact same cubemap texture from a .ktx or .ktx2 file into either OpenGL or Vulkan, so far without success. How can I achieve this. I tried setting up a RH coord system in Vulkan and loading an OpenGL cubemap but, of course, cubemap face selection is unaffected by the viewport transform so +ve y will always select the posy face which in an OpenGL cubemap is the sky but in Vulkan should be the ground.

The requirements for cubemap images are specified quite exactly in the OpenGL standard. The reasoning for using these orientations is apparently that this is how Renderman did it. And since they “invented” cubemaps, OpenGL just did what they did.

What does “the ground truth” mean?

The biggest difficulty with all of this is knowing where you’re starting from. A KTX file just contains a bunch of image data. How was that image data created? Was the tool which generated it used correctly, or was it misconfigured? Etc.

It’s really best to start from a set of 6 loose images with a known orientation, which you want to provide as a cubemap. For each of the 6 images, you should know which face you want it to go onto, what the orientation in texture space is relative to the destination cube space, and so forth.

Until you have that starting point, it’s basically a guessing game as to how to go about fixing any cubemap orientation problem.

What does “the ground truth” mean?

It means I have 6 .jpg files with images taken at a location I know personally so I know exactly what the real world orientation (i.e. ground truth) is. The files are correctly identified as posx, negx, posy, negy and posz w.r.t. the file labelled negz, taking the latter as the front.

I am making the .ktx files. I know exactly what is in them. If I create a ktx file containing a cubemap with the 6 images all having a lower left orientation then the rendering of the skybox (my scene background) is upside down and flipped left to right.

I’m not trying to fix an orientation problem. I am trying to understand the reasoning to see if I am missing something and in the hope that properly understanding it will help me find a solution to rendering the same .ktx cubemap file in OpenGL and Vulkan without having to flip or swap images.

Are they “correctly” identified as such? Because if you need to horizontally flip the X/Z images as you’ve described, odds are good that your coordinate system is inverted.

If we take the NegZ face as “front”, then directly to the left should be the PosX axis. Cubemaps are in a left-handed coordinate system.

This diagram shows the way the axes work in OpenGL:

Cubemap axes

You would only need to do the horizontal flipping you suggest if you’re expecting a right-handed cubemap space.

I posted about this on Facebook about 6 months ago. I just made the post public in case it is helpful to anybody. The tl;dr is that the image layout came from RenderMan, which has a left handed coordinate system and whose image coordinates have an upper-left origin. With those two things, the cube map layout is much more intuitive. It doesn’t fit well with OpenGL conventions though.

1 Like

Thanks @cass. I had come to the conclusion from looking at the equations in the spec. that cube map images had to be top down and the faces labelled in a left-handed coordinate system. But I was struggling to believe it because (a) it is so bizarre to do this in a API which is otherwise right-handed and (b) not one of the many cube map samples and tutorials I looked at mentions this bizarre behavior.

I’m still struggling with left and right being switched in my final result. The input images are correctly labelled in an LH system. That is, with +y being the sky and -y the ground, if you look in the real world scene towards what is in the image labelled +z, what is on your right is in the image labelled +x and on your left is in the image labelled -x. The images are being uploaded to the same labelled texture targets. All images have upper left origins.

I am using the skybox cube’s vertex coordinates as the texture coordinates. I’m not performing any transform on them. I am starting with an identify matrix for the view matrix so the initial view is looking along the -z axis and what I see is the image in the -z texture but it is flipped left to right. If I turn the view around to look at +z it too is left-right flipped as is every image. When looking at -z the image on the left is the -x image and on the right +x but what is on the left should be on the right because it was on my left when looking towards +z in the original images.

I’m currently fixing this by multiplying the x texcoord by -1. I wonder if I should load the images according to the OpenGL coordinate system. That is should I exchange the +z and -z images. If so then my theory about a LH coordinate system for the faces is wrong.

I wonder if I should load the images according to the OpenGL coordinate system. That is should I exchange the +z and -z images. If so then my theory about a LH coordinate system for the faces is wrong.

This doesn’t work. The images are still mirrored plus it introduces a lot of other problems. I’ve reverted to following the original labelling with the -1 scale on x to fix the flipping.

Here is my vertex shader:

"layout (location = 0) in vec3 inPos;\n\n"

"out vec3 vUVW;\n\n"

"out gl_PerVertex\n"
"{\n"
"  vec4 gl_Position;\n"
"};\n\n"

"void main()\n"
"{\n"
"  vUVW = inPos.xyz;\n"
"  gl_Position = (ubo.projection * ubo.modelView * vec4(inPos.xyz, 1.0)).xyww;\n"
"}\n";

And the fragment shader.

"uniform samplerCube uSamplerColor;\n\n"

"in vec3 vUVW;\n\n"

"layout (location = 0) out vec4 outFragColor;\n\n"

"void main()\n"
"{\n"
  "  outFragColor = texture(uSamplerColor, vUVW);\n"
"}\n";

It couldn’t be much simpler.

It looks like the skybox cube texture coordinates need to be transformed to a left hand coordinate system for the cube mapping to work correctly. That is what x * -1 is doing. z * -1 works equally well. Can anyone confirm this or do I have just have some bug in my program?

Skyboxes are in a left-handed coordinate system. So if you’re providing a vector direction from a right-handed system, it will be “backwards” in some axis.

Skyboxes are in a left-handed coordinate system. So if you’re providing a vector direction from a right-handed system, it will be “backwards” in some axis.

I figured out what is going on and think “skybox” is ambiguous in the above. It is the cube map that has a left-handed coordinate system. To satisfy the face selection tables and (s, t) calculating equations in the OpenGL and Vulkan specs. (which are identical) all images need an upper left origin and they must be arranged according to a left-handed coordinate system with +Y up. That means sky at +Y and if you’re facing +Z, -X should be on your left and +X on your right.

Now the coordinates of the cube you are rendering as the skybox (see the ambiguity?), which will be used to sample the cube map are in the API’s coordinate system. In OpenGL this is a right-handed system. To transform to the cube map’s left-handed system you must scale the Z coord by -1. Failure to do that means the scene will be a mirror image of what it should be. The problem I started this with.

Vulkan has a left-handed coordinate system but +Y is down. So the cube coordinates need to be rotated 180 degrees around the X axis to transform them to the cube map’s system before sampling.

It is clear that a lot of people have trouble with this. Every OpenGL cube map tutorial and sample I looked at and even a bachelor’s degree thesis use the cube coordinates untransformed so the images are mirrors of what they should be.

It’s not much better among Vulkan users. I have found hacked .ktx cube maps with all the images having a bottom-left origin and the +Y and -Y faces swapped. This of course gives the cube map a right-handed coordinate system but at least the images are the right way up !!! when sampled using untransformed cube coordinates. Sadly such samples suffer from the mirror image problem when the l-h coord is used to sample the r-h cubemap. The simple rotation mentioned above is the correct solution and does not require hacking the cube map textures.

Do they? It seems to me that this only happens when you use a cubemap to directly map a texture onto a cube. If you’re using a cubemap for something like projective texturing (where you’re using a direction rather than a position, which is what cubemaps are for), then this problem doesn’t exist. Personally, I would use an array texture for a skybox, not a cubemap.

this only happens when you use a cubemap to directly map a texture onto a cube.

All the samples I found when searching for OpenGL cubemap sample or tutorial do this. I didn’t stumble across one doing projective texturing.

Personally, I would use an array texture for a skybox, not a cubemap.

Then you won’t have seamless filtering across the edges so you will likely see them.