Terrible aliasing.. how to fix?

Ok so I’m rendering a large field of billboarded grass and towards the distance the edges of the grass are not smooth at all. Worse yet when I move there’s tons of pixels dancing all around in the distance… it looks just hideous far away and I can’t even discern the motion of the grass through the pixely haze.

Since all of the filtering methods I know of are not easy to implement ( for example to use GL_POLYGON_SMOOTH I need to depth sort all of the grass ) and fullscreen antialiasing sounds even more daunting, I just wanted to make sure there wasn’t some built-in features of opengl that can smooth the pixels better.

Also, would texture filtering methods like anisotropic filtering help at all here? If there isn’t an easy built-in opengl answer I’d appreciate any pointers to what filtering methods work best, and how I should implement them (i.e. should I use fragment shaders?).

Sounds more like z-fighting. What are your near and far clipping planes and what kind of pixelformat do you use?

Anisotropic filtering will help avoid blurriness when seeing a texture polygon at a sharp angle (ie. not facing the camera), so it is absolutely not what you need. It can help for road lines converging toward the horizon, etc.

GL_POLYGON_SMOOTH ? well not very useful with billboards. Only if your gras is made of many thin triangles.

How do you make you billboards transparent ? With alpha testing ? Try alpha blending instead. But you will have to sort from back to front. However, you can pre-sort in 4 directions, and only choose wich order you need at runtime.

ie :
case 1 :

5678 -
  | camera in this quarter

case 2 :

- 8765 -

etc. Hope you get the idea.

First of all I should clarify what I meant by billboarded… the actual polygons of the grass are rendered so they always face the camera, so each blade of grass isn’t a quad with alpha values, its just about 10 polygons always facing the camera.

As far as z fighting its possible because there is a huge amount of overlapping grass, but I set my depth buffer to 24-bit and scrunched in the near/far planes as much as possible, and that didn’t improve it at all.

Thanks for clarifying why anisotropic filtering is not what I want though… it makes sense now that I think about it.

So in summary I don’t think its z-fighting, and I guess I mislead you by saying “billboard”, what I really meant was several tris per blade of grass, but they always face the camera.

Any suggestions now that that’s cleared up?

That’s also kind of billboard. Are the grasses far from each other so when they rotate around their axis, they won’t overlap each other?

Its a dense field of grass, so if you mean are there blades of grass obscured by other blades of grass? Yes… many. But there is still plenty of breathing room between adjacent blades, so none of them physically intersect with each other. I’d put an image up somewhere but I don’t have any webspace.

But let me make clear the grass is NOT transparent in any sense, they’re just plain old polys that always face the camera.

If the tiny blades in the distance alias, it means they are too small to be rendered completely because their area misses the rasterization spots sometimes and nothing is drawn.
If you keep your grass rendering this way the only method to get it better is to increase the sampling rate of the rasterizer.

That is, it gets better in higher screen resolutions or with full screen antialiasing.
If you don’t want to program for it, just try to force it in the display control panel of your vendor’s graphics board.

For far away grass blades it would be better to use some sort of billboard/impostor technique so that you can use mipmaps containing prefilterd grass blades. The detail is not important that far away you only need to make sure it looks continuous.

Ok, thanks, I’ll consider the filtered mipmap idea at a distance. So there’s really no other way to draw fractional values of the pixels so they look smooth other than going all-out and doing FSAA?

Interesting… I just hacked GL_POLYGON_SMOOTH (none of the depth sorting is right) and the grass looks fantastic, so that’s definately my problem. However its ridiculously slow… I really would hope there’s a better more comprehensive way.

Without a screenshot, it’s hard to say, but to me that sounds like you aren’t using MIP mapping, and so are under-sampling your textures. A screenshot would help.

Sadly it only truly looks bad when the camera moves, but just imagine the entire field from mid range to far range “shimmernig” and pixels dancing around when the camera moves.

I am using mipmapping btw, but even without texturing the grass aliases terribly with the background. With some help from this forum and some experimenting it seems to me I can eliminate the possibility of any sort of depth fighting or texturing problems… it has to be the actual smoothness of the polygons themselves.

So… what can I do about it?

Aah. Thanks for the picture. Yea, MIP mapping won’t help here. It looks like you are undersampling your geometry – that there are just too many blades of grass. I’m not sure if there’s a good solution :-/

If you can’t afford the POLYGON_SMOOTH nor the FSAA, then you have to go for transparent billboarded quads, each with a bunch of grass blades with alpha.
All the grass you see in games (think of Serious Sam 2, FarCry) is done with transparent quads.

For TrackMania though, they have very clever pixel shader on the ground texture, that give a very convincing effect, especially when moving through it. See the foreground in this screenshot
By the way, this game features offset (or parallax) bump mapping too !

Just a guess and maybe not very helpfull but maybe try inreasing the screen resolution and color. I had similar problems and this helped a little if it is an option.