When optimizing what triangles to draw, I check if the corners of the triangle is inside the view frustum. That doesn’t work well when the triangle is big as the surface can be visible even if the corners are not.
Is there a recommendation how to test this?
One idea I have is to test if any side of the triangle is inside the view frustum. Given two points, is there an easy way to find out if the line between these two points is going through the view frustum?
A frustum is the intersection of 6 half spaces. A triangle is definitely not in the intersection if there is at least one half space that does contain at least one vertex.
This test is conservative in that one can construct cases where it does not reject a triangle even though it is not in the frustum.
If you want to do frustum culling you should not test individual triangles, but instead whole objects (i.e. thousands of triangles) otherwise the time spent testing triangles is much larger than what you save by submitting as few triangles as possible to the graphics card.
To expand on what Carsten said, you generally test against larger objects than triangles. Groups of triangles, objects, are tested based on some kind of bounding volume. That is, a volume that definitely encompasses all of the triangles in that volume.
An axis-aligned bounding box, for example. Basically, this is just a box with the maximum width, height, and depth of the object, centered at its geometric centerpoint. Bounding spheres (compute the geometric center of the object and the largest distance to one of its vertices) are also used. In some cases, axis-aligned vertical cylinders are used.
All of these shapes are much simpler to test against view frustums. You should do some Googling for “frustum culling” to find out more.
I do test against larger objects. In this case, I do have a (big) cube that rather perfectly contains a large set of other objects. When I mentioned a triangle in the question, it was to generalize the question.
I already did some searching on the web for frustum culling, but most of them are applicable for objects inside the frustum. I didn’t search thoroughly, so I’ll have another look…
Easiest way seems to be to test if all corners are on the same outside of any of the planes of the frustum. Maybe it is what Carsten said?
To perform intersection test between convex objects accurately, you have to perform separating axis overlap tests. They are mainly used for collision detection, but for view-frustum culling as well: http://en.wikipedia.org/wiki/Separating_axis_theorem