Getting the normal of a plane

I have a camera that is oriented using two angles, a rotation around the y-axis (looks left/right) and a rotation around the x-axis (looks up/down). I need to find the equation for the four planes of the view frustum (not including the near and far). I think the most useful form of the plane equation is as follows:

Ax + By + Cz + D = 0

Where <A,B,C> is the unit normal pointing away from the plane and D is the distance between origin and the closest point on the plane.

I understand that you can get the unit normal by using the cross product on two vectors and scaling it to a size of one.

But how do I get two points on the plane if all I know is that the camera is rotated around the x-axis and the y-axis. I also know the distance to the far plane of course.

Simply create the frustum planes for the default GL view (looking down the negative z-axis, with y going up), then transform them with the camera matrix.

I think I understand what you are saying and how to implement it but tell me if the following is correct. What I want to do is get the plane equation for each side of the view frustum (I will only get top, bottom, left and right because the first thing I will check is the distance to the object which in most cases in a large world should be an easy out). Since each plane is going to intersect at (0,0,0) the equations will be Ax + By + Cz = 0 where <A,B,C> is a unit normal pointing directly perpendicular to the plane.

So what I need to do is calculate the four plane equations in a “standard” position which in my case we will say is looking directly down the x-axis. This will be done in the initialization function before normal execution starts. So for each plane I will get one normal. Then when I want to figure out a different normal I need to rotate the normals for the “standard” position by however much the camera is rotated. The thing I really need to know is how to rotate that point to the correct position. The following is what I think I need to do but tell me if I am wrong:

A = cameras_y_rotation
B = cameras_x_rotation

//First get a matrix that represents
//both a rotation around the x-axis and a rotation
//around the y-axis

|cosA 0 -sinA 0 |
|0 1 0 0 |
|sinA 0 cosA 0 |
|0 0 0 1 |

|1 0 0 0 |
|0 cosB sinB 0 |
|0 -sinB cosB 0 |
|0 0 0 1 |


|cosA (-sinA)(-sinB) (-sinA)(cosB) 0 |
|0 cosB sinB 0 |
|sinA (-sinB)(cosA) (CosA)(cosB) 0 |
|0 0 0 1 |

//So the I multiply the normal for each of the planes that
//I precalculated by this matrix and I get the following:

x = normal_x
y = normal_y
z = normal_z

rotated_normal_x = x(cosA) + z(sinA)
rotated_normal_y = x(-sinA)(-sinB) + y(cosB) + z(-sinB)(cosA)
rotated_normal_z = x(-sinA)(cosB) + y(sinB) + z(cosA)(cosB)


Of course when I calculate the rotated normals for each plane cosA, sinA, cosB, sinB will be the same for each plane so I can calculate those only once for all four planes, and I am sure there are some other optimizations I will think of. But in general is this correct or is there a faster way?

After testing my idea above I realize it is wrong. But the problem is that I do not know what you mean by the camera matrix or how to construct it. Thanks for your help so far.

You’re on the right track, but it’s important to start with the “default” GL view for this to work. Think of it as transforming the frustum with the camera, so that they’re attached.

The camera matrix itself is simply the modelview matrix after all the camera transforms have been applied. You can grab it with glGetFloatv( GL_MODELVIEW_MATRIX, MV ), for example.

Here’s one way to create a frustum in the default GL view:

void createFrustum( float FOV, float zNear, float zFar, float MV[16], Plane frustum[6] )
    // Default frustum looking down -z axis
    float a = FOV * M_PI / 360;
    float c = cosf(a);
    float s = sinf(a);
    Plane planes[4] = 
	Plane( +c, +0, -s, +0 ), // Left
	Plane( -c, +0, -s, +0 ), // Right
	Plane( +0, +c, -s, +0 ), // Bottom
	Plane( +0, -c, -s, +0 ), // Top
    // Transform into world-space
    for( int i = 0; i < 4; i++ )
	frustum[i] = transformFrustumPlane( MV, planes[i] );

And here’s a way to transform the plane into world-space:

Plane transformFrustumPlane( float M[16], Plane P )
    float x = P.x*M[0] + P.y*M[1] + P.z*M[2];
    float y = P.x*M[4] + P.y*M[5] + P.z*M[6];
    float z = P.x*M[8] + P.y*M[9] + P.z*M[10];
    float w = P.x*M[12]+ P.y*M[13]+ P.z*M[14] + P.w;

    return Plane( x, y, z, w );

There’s a section in the Red Book that talks about transforming normals and planes if your interested in some of the math behind this. Normally you need to transform normals with the inverse transpose of the matrix that transforms points. But in the case of the camera matrix (already the inverse of the matrix that orients the camera in the world), we can take a short-cut, or two.

I store my planes as Ax + By + Cz = D, so I hope I didn’t miss a sign somewhere after converting.

Hope this helps.

What I was thinking is I would keep the plane equations relative to the origin. This way in the equation Ax + By + Cz = D the D is always 0 and does not need to be calculated. Then I just need to find the position of the point I am testing relative to the camera. So if the camera is at (10,0,10) and the point I am testing is at (30,0,100) the relative position of the point would be (20,0,90). So I would construct the plane equations once per frame and then translate each object relative to the camera and test if it is in the view frustum (sort of like keeping the camera still and moving the world). Or maybe there is some advantage that I do not yet see in finding the exact plane equation?

The strange thing about the plane equation is I have seen it in both forms:

Ax + By + Cz + D = 0
Ax + By + Cz = D

Maybe one was a misprint. I always thought that when you have an equation what you do to one side you have to do to both sides:

Ax + By + Cz (-D) = D (-D) == Ax + By + Cz - D = 0

So I guess that one of those eqations must be wrong. Maybe it has something to do with the fact that D is always positive?

Check these out:

It is also in the form : Ax + By + Cz + D = 0 in the book “Tricks of the 3D Game Programming Gurus” by LaMothe

So I guess that one of those eqations must be wrong. Maybe it has something to do with the fact that D is always positive?
Neither equation is wrong. I simply store the plane distance, whereas many books store the negative of this value (both are perfectly legitimate). The mathematical convention is Ax + By + Cz + (-distance) = 0, where -distance = D. In other words, it makes no difference which sign you prefer, as long as you’re consistent, and you remember to change signs when needed. Store it as you wish.

Here’s a free linear algebra book (PDF):

Many game programming books include some math needed for common operations in 3D. But for real insight into the mechanics of how this stuff works, there’s no substitute for an in-depth look :slight_smile:

By the way, were you able to transform your view frustum?

Thanks a bunch for the pdf book. It is near the top on my list of things to do in the immediate future. I think I figured out how to do what I need to do “in theory”. Now I need to test it to see if it works. Thanks again for all the help.