GLSL determinant() returns an 'undefined' result

I have a little function that should get the intersection point of a ray with an infinite plane.
The function looks like this :

vec3 raytracePlaneI(vec3 s, vec3 a, vec3 b, vec3 o, vec3 d)
  float lU;
  float lV;
  float lT;
  vec3 lN;
  vec3 A = s;
  vec3 B = a;
  vec3 C = b;
  vec3 R = o-A; //translation
  mat3 matrix = mat3(B, C, -d);
  mat3 matU = mat3(R, C, -d);
  mat3 matV = mat3(B, R, -d);
  mat3 matT = mat3(B, C, R);
  float detM = determinant(matrix);
  float detU = determinant(matU);
  lU = detU/detM;
  float detV = determinant(matV);
  lV = detV/detM;
  float detT = determinant(matT);
  lT = detT/detM;  
  if(detU > 0)
    resultColor = vec4(1, 0, 0, 1);
  else if(detU < 0)
    resultColor = vec4(0, 1, 0, 1);
  else if(detU == 0)
    resultColor = vec4(0, 0, 1, 1);
    resultColor = vec4(1, 1, 1, 1);
  return vec3(lU, lV, lT);

In the last part of the function I set the result color to an other color dependent of the value of detU.
I test for >, < and == 0 and it always takes the else part (white color).

Do anyone know whats the error ??

That suggests that detU is NaN. That would imply that at least one element of the matrix is NaN.

Alternatively, something elsewhere in your code may be triggering undefined behaviour, resulting in resultColor not getting assigned.

The problem was that did pow(-x, y) to calculate an argument for the function and this resulted in a NaN.
thank you.

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.