Parallax and environment mapping

I’m trying to do parallax bump mapping and environment mapping but there seems to be something wrong with the transformations or vectors in my shaders. I’m fairly sure the tangent space vectors and the world transform are correct, but when using the following code I get two problems:

a. The parallax effect is not 3d and is ‘swimming’ over the surface with different viewing angles and distorting where it should be offset.
b. The reflections are all incorrect as if the cube map is rotated 90deg and distort/stretch off to infinity at shallow viewing angles.

Can anyone see what I maybe doing wrong?

Vertex Shader


#version 120
attribute vec3 Normal; // Object space normal
attribute vec3 Tangent; // Object space tangent
attribute vec3 Binormal; // Object space binormal
attribute vec3 BumpMultiTexCoord; // Using instead of gl_MultiTexCoord1 

uniform mat4 View2World; // View space to world matrix

varying vec3 view;
varying vec3 normal;
varying mat3 tbnWorldMatrix; // Tangent to world space matrix
varying vec2 bumpTexCoord;

void main(void)
{
  gl_TexCoord[0] = gl_MultiTexCoord0; // Base Texture coords
  gl_Position = ftransform();

  bumpTexCoord.xy = BumpMultiTexCoord.xy; // Bump map coords

  // Calculate view to tangent space matrix
  vec3 t = normalize(gl_NormalMatrix * Tangent);// Tangent object -> view space
  vec3 b = normalize(gl_NormalMatrix * Binormal); // Binormal object -> view
  vec3 n = normalize(gl_NormalMatrix * Normal); // Normal object -> view space
  mat3 tbnMatrix = mat3(t, b, n);

  // Calculate tangent space normal
  normal = normalize(gl_NormalMatrix * Normal); // View space normal
  normal = normalize(normal) * tbnMatrix;

  // Calculate view space view vector
  vec4 vert = gl_ModelViewMatrix * gl_Vertex;
  vec3 viewVec = vec3(-vert);

  // Calculate tangent space view vector
  // ** Reflections work better without the normalize here for some reason? **
  view = normalize(viewVec) * tbnMatrix; 

  // Calculate tangent to world matrix
  tbnWorldMatrix = transpose(tbnMatrix) * mat3(Cam2World);
}

Fragment Shader


#version 120
uniform sampler2D BaseTexture;
uniform samplerCube CubeMap;
uniform sampler2D BumpMap;
uniform mat4 View2World;

varying vec4 col;
varying vec3 view;
varying vec3 normal;
varying mat3 tbnWorldMatrix;
varying vec2 bumpTexCoord;

void main(void)
{
  vec2 cBumpSize;
  cBumpSize.x = 0.04200002;
  cBumpSize.y = -0.0299999;

  // Find parallax offset
  float height = texture2D(BumpMap, bumpTexCoord.xy).a;
  height = height * cBumpSize.x + cBumpSize.y;

  vec3 viewVec = normalize(view);
  vec2 newBaseUV = gl_TexCoord[0].xy + viewVec.xy * height;

  // View vector in tangent space for fragment
  vec3 V = normalize(view);

  // Calculate reflection vector of view vector around normal
  vec3 N = normalize(normal);
  vec3 R = reflect(V, N);

  // Transform reflected vector back into world space
  R = tbnWorldMatrix * normalize(R);

  // Look up into the env_map after transforming into world coordinates
  gl_FragColor.rgb = textureCube(CubeMap, R).rgb;

  // Apply parallax mapping
  // gl_FragColor.rgb = texture2D(BaseTexture, newBaseUV).rgb;

  gl_FragColor.a = 1.0;
}

Some of this code is from the Shader Designer examples, but I seem to be doing something wrong.

Thanks in advance.

First few comments:
The viewVec should be not normalized in the vertex shader, otherwise the interpolated direction will be wrong.
The calculation of the tangent space normal is not necessary. Because of how the tangent space is constructed, it is always the (0,0,1) vector.

Are you sure that the offset calculated by the paralax is sufficiently small? If it is not, the surface might look distorted.
How abrupt are height changes within the height map? With the single sample paralax lookup you are using, significant changes of the height will look bad.

b. The reflections are all incorrect as if the cube map is rotated 90deg and distort/stretch off to infinity at shallow viewing angles.

I think that you are multiplying the reflected vector with the tangent->word matrix in bad order.

Thanks Komat,

I’ve removed the nomalise from the viewVec which has helped the reflections, but I am still having problems with the way the viewVec is calculated. I’ve tried a small paralax offset and height but the offset bit are still moving incorrectly. In desperation I tried messing with the viewVec in the new code below by swapping its components and its better, but not right.

When rotating about one axis (vertically) the parallax effect is shifting horizontally. Any ideas?


#version 120
attribute vec3 Normal; // Object space normal
attribute vec3 Tangent; // Object space tangent
attribute vec3 Binormal; // Object space binormal
attribute vec3 BumpMultiTexCoord; // Using instead of gl_MultiTexCoord1 

uniform mat4 View2World; // View space to world matrix

varying vec3 view;
varying vec2 bumpTexCoord;

void main(void)
{
  gl_TexCoord[0] = gl_MultiTexCoord0; // Base Texture coords
  bumpTexCoord.xy = BumpMultiTexCoord.xy; // Bump map coords
  gl_Position = ftransform();

  // Calculate view to tangent space matrix
  vec3 t = normalize(gl_NormalMatrix * Tangent);// Tangent -> view space
  vec3 b = normalize(gl_NormalMatrix * Binormal); // Binormal -> view
  vec3 n = normalize(gl_NormalMatrix * Normal); // Normal -> view space
  mat3 tbnMatrix = mat3(t, b, n);

  // Calculate view space view vector
  vec4 vert = gl_ModelViewMatrix * gl_Vertex;
//  vec3 viewVec = vec3(-vert);
  /**** This is better but still not right ****/
  vec3 viewVec = vec3(vert.z, vert.y, -vert.x);

  // Calculate tangent space view vector
  view = viewVec * tbnMatrix; 
}

The swizzling should be not necessary and it likely introduced the x shift you are seeing.

How the surface reacts to rotations with the original “vec3 viewVec = vec3(-vert);”? Are you sure that the tangent and binormal correspond to u and v directions within the base texture?

You can also use the GLSL Devil shader debugger to see what are values of the individual variables.

glslDevil looks quite nice, until you start to use it. Values not displayed in ‘value’ column on watch window. Crashes when trying to debug a frag prog.

Thanks for pointing that out, the tangent and binormal were not corresponding to u and v. I’ve swapped them round and its much better now. I’m not taking the handedness into account when calculating the binormal but I think I can figure that out.

There’s only one problem which is still confussing me regarding the reflection into the cubemap. I’ve swapped the tangent->world multiply over and it works perfectly when using the varying normal to give a planar reflection.

But when I try and plug-in the BumpMap’s normal the reflection just gets completely stretched out and converges at a single point. I thought that ‘normal’ and ‘bump’ would be in the same space? Does the reflection calculation need to be done in world space?


#version 120
uniform samplerCube CubeMap;
uniform sampler2D BumpMap;

varying vec3 view;
varying vec3 normal;
varying mat3 tbnWorldMatrix;
varying vec2 bumpTexCoord;

void main(void)
{
   // View vector in tangent space
   vec3 viewVec = normalize(view);

   // Get bump map normal
   vec3 bump = texture2D(BumpMap, bumpTexCoord.xy).rgb * 2.0 - 1.0;
   bump = normalize(bump);

   // Calculate reflection vector around normal in tangent space
   vec3 N = normalize(normal);
//   vec3 R = reflect(viewVec, N); // This works for planar reflection
   vec3 R = reflect(viewVec, bump); // This does NOT work for bumped reflection

   // Transform reflected vector back into world space
   R = normalize(R) * tbnWorldMatrix;

   // Look up into the cube map
   gl_FragColor.rgb = textureCube(CubeMap, -R).rgb;
   gl_FragColor.a = 1.0;

I can see that now because doing gl_FragColor.rgb = N.xyz; renders everything blue.
But, if I do R = reflect(viewVec, vec3(0,0,1)) its incorrect and not the same as using N even though N is (0,0,1). VERY strange.

Thanks again.

I only tried it on one demo from the Nvidia SDK where it appeared to work although if I remember correctly, getting the watched value to show was not intuitive.

Is the sampler properly configured? What happens if you simply draw the result of sampling of the BumpMap? What happens if the bumpmap is entirely filled with (128,128,255) so the expansion results to the (0,0,1)?

I can see that now because doing gl_FragColor.rgb = N.xyz; renders everything blue.
But, if I do R = reflect(viewVec, vec3(0,0,1)) its incorrect and not the same as using N even though N is (0,0,1). VERY strange.

In what way it is incorrect? What happens when you set the normal varying in the vertex shader explicitly to the (0,0,1)? You can also draw the reflected vector as color to see possible differences in both cases. What GPU do you have, ATI or Nvidia?

When just sampling the BumpMap its mapped correctly onto the polys. I created a (128,128,255) bumpmap and get exactly the same results as hardcoding vec3(0,0,1) in the fragment shader.

It actually seems alright when set in the vertex shader and then interpolated, its just setting it in the fragment shader thats wrong (but (0,0,1) interpolated to (0,0,1) is surely = (0,0,1))?

To illustrate it here are some screen shots. The first two are setting gl_FragColor to R using either N or bump in the reflect function:

The second two are sampling the CubeMap using R calculated both ways:

My card is a NVidia Quadro FX3500. I was also getting odd results when using more texture samplers (for gloss, detail and diffuse env) but I think it was running out of resources, so I removed them for now.

Additional thing to check are the varyings. It is possible that the compiler does something wrong with the assignment to the hw interpolators and some values become aliased (this might change depending on which inputs are used by the shader).

Things to try:

  • [li]Modify the shader to use only generic varyings (no gl_TexCoord ones).[]Modify the shader to use only builtin varyings (gl_TexCoord).[]Try to pass the matrix as four vec4 varyings instead of matrix one. (This can be also checked using both ways)

I was also getting odd results when using more texture samplers (for gloss, detail and diffuse env) but I think it was running out of resources, so I removed them for now.

Check that you assign disjunct values to the sampler uniforms. This looks like some of them (e.g. cube and 2d texture) might be assigned to the same texture unit.

It’s now working! I’m not passing four vec4’s but now passing tbnWorldMatrix as a mat4 instead of mat3. It looks like the interpolation of the matrix required all four rows/columns to get the correct result.

The only downside of this is that it’s using up more resources to pass the extra matrix elements now so I have to be careful how many uniforms/varyings I’m using, but it’s giving correct per-pixel environment mapping now.

Thanks again Komat for pointing me in the right direction. :slight_smile: