Polygon antialiasing not working

I’m getting mad with this. I want to make a easy to render selection menu of a game look antialiased with polygon antialiasing. I have been looking for info for a lot of time, including a lot of samples of code and trying what the red book says about doing glEnable(GL_POLYGON_SMOOTH); and glBlendFunc(GL_SRC_ALPHA_SATURATE, GL_ONE); but I’m beginning to think that my video card doesn’t support it (GeForce 4 MX 440 with detonator drivers) or that I’m just too stupid to make this work. Please, help me.

Two possibilities, did you glEnable(GL_BLEND), also with saturate alpha you need a framebuffer with destination alpha. This is requested via the pixelformatdescriptor.

Well, I have this code in my draw_scene() function:
glPolygonMode(GL_FRONT, GL_FILL);
With this I can’t see anything in my scene. If I kill the glEnable(GL_BLEND); line I can see my scene but with those d**n jaggies. The last line is an antialiaser from NeHe that I downloaded and implemented in my code, but that doesn’t work too. I didn’t understand much the second part of your message, I mean everything apart from the glEnable(GL_BLEND); request thing.

You have no destination alpha buffer. And I think that gives you 1.0 for alpha which is fully saturated and so you don’t draw anything.

With the saturate alpha algorithm you draw front to back and eliminates accumulates contributions to the alpha buffer. When the alpha buffer is one full fragemnt coverage is assumed. You start out with full coverage with no alpha buffer.

That’s your problem, look at the pixelformatdescriptor.

There are two ways to get antialiased primitives-- SMOOTH, and MULTISAMPLE. The two paths are MUTUALLY EXCLUSIVE-- if you enable multisample, all SMOOTH hints are disabled. You should read the GL spec.

For regular smoothing, enable blending and the proper hints.
For multisample, you need to create a pixel format with multisample buffers.

Your GF4MX should work either way, however the resulting AA quality will be very different depending which primitives you use and how many multisample buffers you create. Also, on some cards (Radeon 9600, 9800) the regular polygon smoothing does not work, the hardware support has been removed.

See pictures comparing antialiasing quality across different cards and FSAA modes.

Ahh… never noticed the multisample enable there. To use multisample AA you need a multisample framebuffer.

Look here to see how to use multisample antialiasing together.

The two kinds of antialiasing are in theory exclusive. However, I was getting BOTH of them to work on a GeForce2Ti tweaked as a Quadro2Pro. I’m sure about that, the quality improvement for the AA lines + 4x FSAA was obvious.
I believe that the OpenGL FAQ should be more detailed about the polygon antialiasing, specifiying that you have to draw on black transparent background and draw the desired background last.

Well, I have taken the red book and did exactly what it says about polygon smooth antialiasing and, as Tzupy says, set the clear color with 1.0f alpha and, frankly, comparing to another supposingly antialiased scenes, looks good but the same that I had in the beginning. I mean that I am not sure if I really have my polygons antialiased. I also killed the multisample aliasing thing. Anyway, should a game have antialiasing enabled in the game menu

The big price you pay for saturate alpha rendering is the required front to back polygon sort. For that reason alone it is difficult to implement and has a significant performance cost.

Almost nobody uses it for any kind of complex 3D scenes in ‘real-time’ AFAIK, but it is much higher quality than multisample because it uses fragment alpha for coverage blending giving you much better quality than any multisampel scheme. (Provided you gamma correct but that’s another story)

[This message has been edited by dorbie (edited 12-29-2003).]