Could someone please explain what’s wrong here. I’m trying to use a stencil buffer.
First of all, I read the relevant parts in the GL specs v1.4 and I searched the forum with the keyword “stencil” and browsed and read most of the relevant topics, but that could not help me so far… I know there’s lots of stencil topics and with some regret I must post yet another one… here goes:
I use GLUT for setting the window:
glutInitDisplayMode(GLUT_SINGLE | GLUT_RGBA | GLUT_ALPHA | GLUT_STENCIL);
I assume the stencil buffer is initialized fine, as after the window is created,
glGetIntegerv(GL_STENCIL_BITS, &iStencilBits);
gives me the value 8.
The pixel format is RGBA, 8 bits/channel, 32 bits. Alpha is used in a not-so-traditional blending scheme (more on that further down).
I draw some quads into the (blank) framebuffer, hoping to get the stencil buffer “pixels” incremented by one every time a pixel is drawn. So, most of the stencil buffer should be zero, with the value 4 in a pixel position where four overlapping quads are drawn, for example.
Note that there is no Z-buffer and depth testing is disabled with
glDisable(GL_DEPTH_TEST); // Depth testing disabled.
Should depth testing be enabled? I can’t see any change if I enable it.
I also have blending enabled with
glBlendFunc(GL_ONE, GL_ONE);
glEnable(GL_BLEND);
The strange thing is that the quads do not accumulate at all (!) Just the last fragments seem to get drawn. That’s really weird. I am aware of the saturation problem at 255 (1.0), this is not an issue.
Here’s snips of my code:
First, the buffers are cleared with:
glClearStencil(0); // Set stencil clearing to zero.
glClear(GL_COLOR_BUFFER_BIT | GL_STENCIL_BUFFER_BIT); // Clear the buffers.
Then I try to setup the stenciling appropriately, this is where the bug lies (I hope 
- I don’t think I have quite grasped the stenciling thingy, it really seems very non-intuitive to me.
Something like this:
glEnable(GL_STENCIL_TEST); // Enable stencil testing.
glStencilFunc(GL_ALWAYS, 1, 255); // Always passes, ref 1, mask all bits.
glStencilOp(GL_INCR, GL_INCR, GL_INCR); // Increment stencil with every pixel that gets drawn.
glBegin(GL_QUADS);
… draw some quads here
glEnd();
When all quads are drawn, I read the stencil data into a buffer, like this:
glReadPixels(0, 0, iWinWidth, iWinHeight, GL_STENCIL_INDEX, GL_UNSIGNED_BYTE, pTempStencilData);
The result is all zeroes.
The stencil buffer itself should be OK, I just don’t know how to increment the values by one for every fragment drawn.
The code snippet:
glClearStencil(7);
glClear(GL_STENCIL_BUFFER_BIT);
glReadPixels(0, 0, iWinWidth, iWinHeight, GL_STENCIL_INDEX, GL_UNSIGNED_BYTE, pTempStencilData);
gives the pTempStencilData buffer full of sevens so no problems there.
Another question: whats the difference between
glEnable(GL_STENCIL_TEST); and
glEnable(GL_STENCIL); ???
#define GL_STENCIL 0x1802
#define GL_STENCIL_TEST 0x0B90
…?
How should I set up glStencilFunc() and glStencilOp(), and (if appropriate) the depth sorting stuff?
What’s the relevance of glBlend(GL_ONE, GL_ONE) not working with this setup?
Thanks for any help,
Andru
