the description claims you specify the 4 coefficients of a new clipping plane,

Ax + By + Cz + D = 0

Fair enough. But in what coordinate system is this plane constructed? Furthermore, shouldn’t we get to say which side of the plane is in, and which is out? glClipPlane seems to include no parameters that specify which side to throw away.

The coefficients (p1, p2, p3, p4) passed to glClipPlane are given in object coordinates (model space). They are immediately transformed by the inverse transpose of the current modelview matrix, i.e. transformed like normals, to eye coordinates (view space):

(p1’, p2’, p3’, p4’) = (p1, p2, p3, p4) * M^-1

Vertices get clipped if the dot product of the eye coordinates with this vector is negative, i.e. the positive side is in.

p1’ * xe + p2’ * ye + p3’ * ze + p4’ * we >= 0

The inverse tranpose of the modelview matrix? All descriptions I’ve read say simply the inverse. If it is supposed to be in model coordinates, this would make sense:

given p = [p1’, p2’, p3’, p4] = [p1, p2, p3, p4]M^(-1)

and given V_e = M*V_m where V_m is the model coordinates, and V_e is the eye coordinates, then:

§V_e = [p1, p2, p3, p4]M^(-1)M*V_m = [p1,p2,p3,p4]*V_m

the result of that matrix multiplication being a 1x1 matrix who’s one entry is the dot product of the plane coefficients, and the original eye coordinates.

All makes sense! So why the transpose? Is that just a technicality required since openGL stores matrices in collumn major format?

If you consider the homogeneous plane P=t(a, b, c, d) defined by your normal N, and a point V=t(x, y, z, 1) on this plane, you have:

tP*V = 0

then if you consider M the modelview matrix that transform V in V’:

tP * inverse(M) * M * V = 0

M * V is V’

and

tP * inverse(M) = tP’

thus: P’ = t(inverse(M))

Note: t is is the transpose operator

Shouldn’t the last line be: P’ = t(inverse(M))*P ???

anyway, I believe I get it. I was earlier treating P as a 1x4 matrix instead of a 4x1 matrix. Either way works, they just use a 4x1 so we need to transpose the inverse of M as well.

Thanks very much to both of you!

am I right in my suspicion that you could reverse the side thats clipped by using [-a, -b, -c, -d] instead of [a, b, c, d]? Its the same plane, but the dot product should have its sign flipped!

Yes you are right, I wrote it too fast.

In my opinion, it is the same plane and the dot product should be reversed.

You know if a point is on or above the plane if the dot product between its homogeneous coordinates and the homogeneous plane equation is >=0. If the result is negative it is under the plane and the point is clipped

I wrote inverse transpose because the usual order of multiplication in OpenGL transformations is M * v, not v * M. So if you put the matrix first it’s the inverse transpose.

Certainly.