Help with stencil please...

Could someone please explain what’s wrong here. I’m trying to use a stencil buffer.

First of all, I read the relevant parts in the GL specs v1.4 and I searched the forum with the keyword “stencil” and browsed and read most of the relevant topics, but that could not help me so far… I know there’s lots of stencil topics and with some regret I must post yet another one… here goes:

I use GLUT for setting the window:

I assume the stencil buffer is initialized fine, as after the window is created,
glGetIntegerv(GL_STENCIL_BITS, &iStencilBits);
gives me the value 8.
The pixel format is RGBA, 8 bits/channel, 32 bits. Alpha is used in a not-so-traditional blending scheme (more on that further down).

I draw some quads into the (blank) framebuffer, hoping to get the stencil buffer “pixels” incremented by one every time a pixel is drawn. So, most of the stencil buffer should be zero, with the value 4 in a pixel position where four overlapping quads are drawn, for example.

Note that there is no Z-buffer and depth testing is disabled with
glDisable(GL_DEPTH_TEST); // Depth testing disabled.
Should depth testing be enabled? I can’t see any change if I enable it.

I also have blending enabled with
glBlendFunc(GL_ONE, GL_ONE);
The strange thing is that the quads do not accumulate at all (!) Just the last fragments seem to get drawn. That’s really weird. I am aware of the saturation problem at 255 (1.0), this is not an issue.

Here’s snips of my code:

First, the buffers are cleared with:
glClearStencil(0); // Set stencil clearing to zero.
glClear(GL_COLOR_BUFFER_BIT | GL_STENCIL_BUFFER_BIT); // Clear the buffers.

Then I try to setup the stenciling appropriately, this is where the bug lies (I hope - I don’t think I have quite grasped the stenciling thingy, it really seems very non-intuitive to me.

Something like this:
glEnable(GL_STENCIL_TEST); // Enable stencil testing.
glStencilFunc(GL_ALWAYS, 1, 255); // Always passes, ref 1, mask all bits.
glStencilOp(GL_INCR, GL_INCR, GL_INCR); // Increment stencil with every pixel that gets drawn.

… draw some quads here

When all quads are drawn, I read the stencil data into a buffer, like this:
glReadPixels(0, 0, iWinWidth, iWinHeight, GL_STENCIL_INDEX, GL_UNSIGNED_BYTE, pTempStencilData);
The result is all zeroes.

The stencil buffer itself should be OK, I just don’t know how to increment the values by one for every fragment drawn.

The code snippet:
glReadPixels(0, 0, iWinWidth, iWinHeight, GL_STENCIL_INDEX, GL_UNSIGNED_BYTE, pTempStencilData);
gives the pTempStencilData buffer full of sevens so no problems there.

Another question: whats the difference between
glEnable(GL_STENCIL_TEST); and
glEnable(GL_STENCIL); ???

#define GL_STENCIL 0x1802
#define GL_STENCIL_TEST 0x0B90


How should I set up glStencilFunc() and glStencilOp(), and (if appropriate) the depth sorting stuff?
What’s the relevance of glBlend(GL_ONE, GL_ONE) not working with this setup?

Thanks for any help,


that’s very strange. your code seems to be ok. i had problem using glut and single buffering. that’s the only thing i see.
try double buffering and add glutSwapBuffers() at the end of the display func and see.

"Another question: whats the difference between
glEnable(GL_STENCIL_TEST); and
glEnable(GL_STENCIL); ??? "

glEnable(GL_STENCIL_TEST) works.
GL_STENCIL is only used in glCopyPixels.

Thanks yoyo, I’ll try with double buffering. I’ve been using single just for debugging purposes. I’ll report tomorrow whether it had any effect on the buggy side

Thanks Relic, I’ll use the “working” one then… I actually glEnabled() both of them just in case

There’s a question left unanswered still though:
Why doesn’t the blending work ? It’s supposed to sum the fragments up with glBlendFunc(GL_ONE, GL_ONE); … right?

[This message has been edited by Andru (edited 07-01-2003).]

do you have cull backfacing enabled? if so, then maybe the quads are being culled. if their verts are in the incorrect order, then they would not be drawn into any buffer. this could be resulting in both the lack of blending and also the lack of values in the stencil buffer.

Well, it works now… both blending and stenciling.

I switched to double-buffering, first there was no difference but at last I got it working. I don’t know what I did but… it’s not broke anymore

Thanks all