antialiasing transparency

OpenGL OpenGL: Advanced Coding
1

Hi,

I was wondering if this new antialiasing transparency setting that i can now set in the nVidia configuration dialog also works when i set the alpha value of a fragment directly from the fragment shader (without using a texture).

So if i have this shader that draws a disc by zeroing out alpha in the corners of a quad…

uniform sampler2D tex;

void main (void)
{
vec4 color = gl_Color;

// Distance to the centre...
float l_DisX = (gl_TexCoord[0].s-0.5f)*(gl_TexCoord[0].s-0.5f);
float l_DisY = (gl_TexCoord[0].t-0.5f)*(gl_TexCoord[0].t-0.5f);
float l_Dis = sqrt(l_DisX+l_DisY);

if (l_Dis>0.5f)
{
	color.w = 0.0f;
}
else
{
	color.w = 1.0f;		
}

gl_FragColor = color;

}

… would the silhouette of the disc be anti-aliased? I’m trying but it doesn’t really seem to work (it works a bit, but not convincing), might be doing something wrong…

Kind Regards,
Daniel Dekkers

2

I think it’s a way of providing antialliasing when using discard instead of alpha.
try using math instead, to get true antialiased edges use something like this perhaps.

color.w = clamp(pow(l_Dis+0.5,20.0),0.0,1.0);

and then vary the fuzziness with the pow function

3

I think what you want is alpha-to-coverage. It only works when multi-sampling is enabled, by outputting the alpha value (opaque or transparent) to each fragment output (i.e. 4x MSAA = 4 fragments).

GL already provides an interface for this, no need for shader hackery:

glSampleCoverage(1.0, false);
glEnable(GL_SAMPLE_ALPHA_TO_COVERAGE);

4

Hi zeoverlord, thanks for your reply,

That is an interesting effect. But now the discs look like they have a glow when they are close to the camera, and aliasing still occurs when they are distant. I am trying to achieve the appearance of “sharp” discs.

Or… i would somehow have to use the depth value of the fragment (gl_FragCoord.z and/or gl_FragCoord.w) in the formula for the “penumbra”-interval. So that distant discs get a relatively large penumbra compared to the size of the quad as discs close to the camera do. I’ll try that…

5

Hi remdul, I had already tried the alpha to coverage, because i knew they use it a lot in foliage rendering (with transparent textures). But for my case (without the textures) it doesn’t seem to do anything in 2x, 4x or 8x times MSAA. At 16x MSAA it shows a small effect, but not very convincing (not as convincing as performing MSAA on the edges of geometry).

6

When you tried this, did you have the driver setting “Antialiasing - Gamma Correction” enabled? It is enabled by default on recent NVIDIA cards, and unfortunately it applies gamma correction to the output alpha value, which subsequently ruins alpha to coverage. I suggest disabling it always, as the quality improvement in the general case is questionable at best.

This is a difficult thing to deal with, because you can’t normally tell end-users to go into their driver control panel and fix this. If you’re using shaders, you could apply the reverse of the gamma correction function to your output alpha value, but this is not a good solution either, for several reasons.

I don’t know if this is a bug or intentional, but I don’t see the usefulness of gamma correcting alpha values. I think the sRGB texture/framebuffer formats leave alpha untouched?

7

Hi AlexN. With (or without) gamma correction, the Alpha To Coverage gives a dithered result. Especially with large smooth alpha transitions. These “discs” go from alpha 1 in the center to alpha 0 at the edge. I didn’t realize the dithering yet, but it makes sense considering the name. I thought it was a subpixel technique, but it seems to be more like a metapixel technique :slight_smile:

I would like to paste a screenshot image, but that doesn’t seem to be possible.

I think i have to do the anti-aliasing myself somehow in the shader. But thanks for the effort.

8

Yes, sorry, the dithering pattern is to be expected and may even be different when comparing different gpus. The intention is to increase the apparent quality of alpha to coverage, but it looks quite ugly across large gradients. I don’t think you can control the pattern through desktop OpenGL. Newer versions of directx I believe expose the ability to write coverage values directly in the fragment shader, but you may be out of luck in OpenGL.

9

NV_explicit_multisample exposes a way to access individual samples.