Matrix Inversion

Hey everone. In my vertex shader, I reckon I’m not calculating the inverse() of a mat3 properly. I am trying to get the brick shader demo from the orange book to work and I’ve got it to render, but the lighting looks a little off. I would have used the built in inverse() that the book claims exists for >= 1.40, but my nvidia driver complains it doesn’t have it:

warning C7547: extension GL_EXT_gpu_shader5 not supported in profile gp4vp

Whatever that means. So I tried to implement it myself. The code for my vertex shader is as follows. I am confident the fragment shader is ok.

  Name:         brick.glslv
  Description:  Vertex shader to procedurally generate brick texture on model...

// We want at least GLSL 1.50...
#version 150

// We need this for inverse(), but I thought it was already supported in 
//  GLSL 1.40...
#extension GL_EXT_gpu_shader5 : enable

// Enable debugging information...
#pragma debug(on)

// Vertex attributes...

    // Vertex position in model coordinates...
    in vec4 mcVertex;

    // Normal plane / vector in model coordinates. Assume already normalized...
    in vec3 mcNormal;

// Uniform variables...

    // Transformation matrices...
    uniform mat4 ModelViewMatrix;
    uniform mat4 ProjectionMatrix;
    // Light position in eye coordinates...
    uniform vec3 ecLightPosition;

// Constants...

    // Lighting parameters...
    const float SpecularContribution    = 0.3;
    const float DiffuseContribution     = 1.0 - SpecularContribution;

// Outputs to be interpolated for the fragment shader...

    // Intensity of light at this vertex...
    out float LightIntensity;
    // Position of vertex to be interpolated in model coordinates...
    out vec2  mcPosition;

// Calculate inverse, in case built-in not supported...
mat3 inverse(const in mat3 Matrix)
    // Calculate the determinant...
    float D = determinant(Matrix);
        // Singular matrix, problem...
        if(D == 0.0)
            return mat3(0.0);

    // Calculate the transpose...
    mat3 MatrixT = transpose(Matrix);

    // Calculate needed determinants...
    float D00 = MatrixT[1][1] * MatrixT[2][2] + MatrixT[2][1] * MatrixT[1][2];
    float D10 = MatrixT[0][1] * MatrixT[2][2] + MatrixT[2][1] * MatrixT[0][2];
    float D20 = MatrixT[0][1] * MatrixT[1][2] + MatrixT[1][1] * MatrixT[0][2];
    float D01 = MatrixT[1][0] * MatrixT[2][2] + MatrixT[2][0] * MatrixT[1][2];
    float D11 = MatrixT[0][0] * MatrixT[2][2] + MatrixT[2][0] * MatrixT[0][2];
    float D21 = MatrixT[0][0] * MatrixT[1][2] + MatrixT[1][0] * MatrixT[0][2];
    float D02 = MatrixT[1][0] * MatrixT[2][1] + MatrixT[2][0] * MatrixT[1][1];
    float D12 = MatrixT[0][0] * MatrixT[2][1] + MatrixT[2][0] * MatrixT[0][1];
    float D22 = MatrixT[0][0] * MatrixT[1][1] + MatrixT[1][0] * MatrixT[0][1];
    // Assemble matrix of cofactors...
    mat3 MatrixAdjugate;
    MatrixAdjugate[0] = vec3( D00, -D01,  D02);
    MatrixAdjugate[1] = vec3(-D10,  D11, -D12);
    MatrixAdjugate[2] = vec3( D20, -D21,  D22);
    // Calculate the inverse...
    return (1.0 / D) * MatrixAdjugate;

// Entry point...
void main()
    // Variables...
    float   Specular;
    mat4    ModelViewProjectionMatrix   = ProjectionMatrix * ModelViewMatrix;
    mat3    NormalMatrix                = inverse(mat3(transpose(ModelViewMatrix)));

    // Calculate the vertex position in eye coordinates...
    vec3 ecPosition = vec3(ModelViewMatrix * mcVertex);
    // Calculate the normalized vector to the light source from vertex...
    vec3 ecLightVector = normalize(ecLightPosition - ecPosition);
    // Calculate the normal vector in eye coordinates...
    vec3 ecNormalVector = normalize(NormalMatrix * mcNormal);
    // Calculate vector to viewer from vertex (viewer is origin in eye coordinates)
    vec3 ecViewerVector = normalize(-ecPosition);

    // Diffuse light at this vertex is a function of how coincident normal and
    //  the incident light wave are...
    float Diffuse = max(dot(ecLightVector, ecNormalVector), 0.0);

    // Specular light at this vertex is a function of how coincident normal and
    //  viewer are, but don't waste time if surface doesn't even face the light...
    if(Diffuse > 0.0)
        // Calculate...
        Specular = max(dot(ecViewerVector, ecNormalVector), 0.0);
        // Amplify...
        Specular = pow(Specular, 16.0);
        Specular = 0.0;

    // Given the vector from surface to light and normal, compute reflected ray...
    vec3 ecReflectionVector = reflect(-ecLightVector, ecNormalVector);

    // Output light intensity to be interpolated for fragment shader...
    LightIntensity = (Diffuse * DiffuseContribution) + 
                     (Specular * SpecularContribution);
    // Output vertex position in clipping coordinates...
    gl_Position = ModelViewProjectionMatrix * mcVertex;

    // Output vertex position to be interpolated for fragment shader...
    mcPosition = mcVertex.xy;


There is no EXT_gpu_shader5, only EXT_gpu_shader4. Apparently you misspelled this:

#extension GL_EXT_gpu_shader4 : enable

in both your GLSL vertex and fragment shaders. Change to the above.

To check these things, go here:

// We need this for inverse(), but I thought it was already supported in
// GLSL 1.40…
#extension GL_EXT_gpu_shader5 : enable

What makes you think inverse() is offered by EXT_gpu_shader4?

Hey Photon. Changing it to 4 or removing the #extension line altogether leaves me with:

error C7531: global function inverse requires "#extension GL_EXT_gpu_shader5 : enable" before use

As for why I think inverse() was suppose to be supported, it’s listed in table 5.5 of the Orange Book (p137).


Well that’s pretty interesting. I’d file a bug on that. Not only is that not a valid extension, if you look at EXT_gpu_shader4 it says nothing about inverse().

That said, I do see benefit in providing inverse() access to GLSL versions prior to 1.4. Going to 1.3 requires some non-trivial app changes.

As for why I think inverse() was suppose to be supported, it’s listed in table 5.5 of the Orange Book (p137).

Absolutely. I think you’re absolutely right there and that version 140 or 150 should get you that inverse() function. Just check that you’re passing a mat2, mat3, or mat4 as input and assigning to the same type as output.

Hey Dark Photon. I will file a bug report now. Thanks for your help.