Stencil operation question

Sorry if this has been asked before, but I didn’t find the answere in other topics…
I’d like to combine stencil shadows and mirrors. For this it would be great if I could divide the stencil buffer into two parts (8bit): lower 4 bits and upper 4 bits. This can be done using glStencilMask(). But if I set glStencilMask() to only allow writing to the upper 4 bits and I use GL_INCR stencilop, would this increment the upper 4 bits? Or the lower 4 bits (but in this case it would do nothing)…

It’s going to increment the whole 8-bit value by one.

But if you’re masking out the lower then it should just not do anything to the upper 4 bits. Unless it increments THEN masks, that could change the upper bits if the number is larger than 15 in the 8-bit string to begin with. But if it mask’s then increments, then the upper bits should still remain unchanged.


[This message has been edited by SirKnight (edited 10-27-2003).]

Combining stencil shadows and mirrors (or remote portals)…

  • Don’t use the stencil test to mask off the surface of the mirror. Instead, follow these steps.

  • Clear the entire depth buffer (and stencil buffer).

  • Restrict your view frustum to the area that bounds the visible surface of the mirror.

  • Render everything that’s visible through the mirror. Remember that winding orders will be reversed, so invert the front-facing winding direction with a call to glFrontFace().

  • Clear the depth/stencil buffer. Using the glScissorRect() function to restrict the clear area to the visible part of the mirror may speed up this clear, but on some hardware (like ATI) it may force an unoptimal path that will actually slow you down.

  • Begin rendering the normal scene. The first thing you render should be the geometry that represents the surface of the mirror. This can be any shape and will initialize the depth buffer so that objects get rendered in front of or behind the mirror correctly.

  • You’ll probably want to look into using an oblique near plane to auto-clip geometry rendered through the mirror to the mirror’s plane (for free). Using this has a significant precision/usefulness trade-off when the mirror is viewed at highly oblique angles. If you’re rendering any geometry through the mirror that crosses the mirror’s plane, this technique is necessary to avoid artifacts. I believe there is a demo on Nvidia’s site.

The details are very complicated and I apologize for not being able to include them all here. I have had this running for some time and it works extremely well, so I hope to have pointed you in the right direction. I’m thinking this may be a chapter in a future book of mine…

– Eric Lengyel

Thanks for the answers. So my first plan won’t work…
Eric: I knew about that solution, but my problem is that I need the depth buffer contect after rendering the mirror because I need to render the particle systems after the mirrors (to achieve fine blending). But now that I think… Maybe if I first render the mirrors and than everything else it may work. I’ll give it a try. Thanks.

EDIT: sorry, I see you also wrote this.

[This message has been edited by Catman (edited 10-28-2003).]

Hmm, new problem. Eric’s suggestion should work well for non-recursive mirrors. But how can I implement recursive mirrors with this method? Clearing the depth buffer would be a problem I think…

This works for recursive mirrors as well. Each time you render with a different view frustum, you first look for mirrors or remote portals that you need to render (excluding the one that you’re currently looking through, if any). For each visible mirror, recursively render (to some recursion limit to protect against mirrors facing each other recurring forever). Clearing the depth buffer and first rendering the mirror’s surface at each level will give correct results. The key is to render the deepest mirrors first.

– Eric Lengyel

This sounds interesting. I’ll try it…

I was thinking more about your method, Eric. What happens when two mirrors partially overlap? You render the first mirror (closer to the camera), clear depth buffer, draw mirror surface. Then comes the second mirror (partially behind the first), render its content, clear depth buffer. And at this point the first mirror’s depth mask is cleared. When you render the second mirror’s surface it is drawn over the first mirror while it should be behind it. Is this correct or am I getting something wrong?