Concave Objects

Hi all,

I’m attempting to draw a number of shapes from a list of vertices using GL_POLYGON. However I am getting some strange effects when the objects are concave in shape. I checked the openGL programmers guide and it seems to state that polygons have to be convex. If this is the case, how do you draw concave objects ?

Any help would be appreciated !


Break up your non-convex polygons into convex polys. I think you could also use a tesselation object to do that for you, but IMHO thats more work than it is worth. I suggest manually cracking polys that are not convex.

The System is used to generate 3D terrain so it could have a lot of concave objects in it so its not really an option to do each manually.

How would you go about applying the tessalation you mentioned ?

I’m going through the same dilemma. I have a user draw a polygon outline for a ‘floor layout’ in my CAD program, and there are concavities in the poly-list.

How does this tesselation work?


Triangulation is the easiest. Take three adjacent vertices, draw a line between the two that aren’t adjacent to each other, and see if that new edge is (a) inside the triangle, and (b) doesn’t cross any other lines. If so, you can break off those three vertices as its own triangle, and remove the middle vertex from your polygon. Do this until you are left with one triangle. Of course, you don’t have to take the first triangle that works… you could apply other heuristics, such as triangle with smallest area or largest area or closest to equilateral.

GLU tesselation is easy peasy, and very versatile. The OGL red book explains it simply, and has lots of sample code.

I tried gluTesselator, and got it working, even for cross-overs in the contour (combineCallback).

The only ‘snags’ I had were platform-specific. My app runs on Windows XP right now, with Visual C++ under Visual Studio .NET. These are the only ‘snags’ I found:

  1. All of my callbacks had to be defined outside of my class (a typical problem with callbacks), and I had to explicitly define them as ‘__stdcall’ (in definition and implementation) to get them to work with gluTessCallback():

gluTessCallback(pTess, GLU_TESS_BEGIN, (GLvoid (__stdcall *) ()) &beginCallback);

  1. The combineCallback() needs to dynamically allocate memory. Since you don’t know how many times it will be doing this you need to push all of the pointers to the allocated mem-chunks into a globally-accessible ‘list’ (I used an STL ‘vector’). Then you need to ‘free’ (delete) the chunks from the list after gluTessEndPolygon() in your code.

I’m posting this so others who may attempt this don’t get ‘snagged’ too.

Like I said, it works great!

Thanks for recommending the gluTesselator,