Lowest value in the stencil

I just thought of something WRT shadow volumes, and was about to post this on the advanced forum, but considered that it would be a good addition to the next OGL. I have seen a lot of extentions, algorithms and so forth that address a problem with shadows. I think it would be a lot simpler to have the hardware store the lowest stencil value “on the screen”. I have done a bit of scratching around, and have found that if the stencil is “cleared” to some arbitrary value, like 128, then stencil volumes are drawn, the unshadowed areas are going to have the lowest stencil value found. No real complicated algorithm required, but currently if I were to implement this, Id have to pull out the entire stencil buffer, dig thru it, and find the least value… Sounds like that’s more “performance intensive” than I would like. If this were handled in hardware, I can see it being rather fast, especially considering that the hardware doesn’t need to check each pixel, it only has to remember “visited” pixels, and the value which the buffer was cleared to. If there isnt a visited pixel with a lower value than the clearance value… I guess you get my drift.

I can see this method clearing up a lot of difficulty in the shadow realm.

But… The unshadowed areas are the ones with the same stencil value you cleared to. Surely you can remember it yourself too. Unless you do your volumes in some wicked way.

Of course such functionality would have it’s uses, but I don’t remembering the visited pixels only, because often you visit most of the pixels anyway, and if you rewrite some of the pixels with other stencil values, it’ll get messed up. I don’t see how this can help anything.

Last, isn’t this what GL_MINMAX is about? Not sure.


I chould see how this functionality could help, just not on Stencil, but on Color an depth buffer too, if we could get both min and max that is…

in boolean operations with complex objects we need to know the maximum complexity in the scene in order to know how many passes we need… and if we know min and max z depth we could optimize the next frames Z values ( we often know how much an object can move in one frame so we can have a buffer-zone) and we could adjusting colors depending on the min/max of an rendered image…

Think about it this way, if you are inside a shadow volume, then your clear value is the value of the shadow inside. The only reason why you don’t get “negative shadow” outside is because you clear to 0, then decrement an unsigned, it should stay 0 (unless Im way off here…). But, imagine we can test for the lowest value, we’d be able to recognize that “negative shadow” as the real lit area, and would also be able to recognize that our clear value is actually in a shadowed volume.

Basically, we’d never need to know whether the cam is in the shadow or not, it should work correctly.

Yeah, maybe if you don’t handle the near plane, but even then I think there will be areas that are unshadowed yet don’t have the lowest stencil value. There will also be shadowed and unshadowed pixels with the same stencil value, because the near clip plane can cut the shadow polygon in the middle of the screen. With zfail you don’t have to know whether the cam is in the shadow anyway, so I don’t consider this that big a problem anyway.


I don’t think that the lowest value in the stencil is representative in the shadowed areas. What is really representative is the stencil values upper/equal/lower than 128 (or whatever constant) but the lowest value can be 100, 127, 128 or even 135 it doesn’t matter.

Anyway even if the “minimum stencil value” is useful for other algorithms, I don’t think it would be easy to setup a fast pipeline that “keeps in mind” the lowest stencil value of the buffer. What would be needed to store that minimum ? a single value ? it would work only if pixels never overlap, which is a too restrictive case. Otherwise all the stencil buffer should be checked (when the lowest stencil value is overwritten) which is even worse.

[This message has been edited by vincoof (edited 03-09-2003).]