OpenGL ES: Stencil buffers and culling weird behaviour

[dpk]

Hello

I am new to OpenGL and i am trying out few experiments especially with the stencil buffers.
In my code, I have set the front and back stencil buffers separately using glStencilFuncSeparate to 0x5 and 0xC respectively(GL_ALWAYS as parameter to the function). glStencilOpSeparate is set for front and back states to GL_REPLACE for dppass. I have also ensured that the depth and the culling are disabled when I setup both the stencil buffers.

Next I try to render a cube with depth, front culling and stencil test enabled. I am now using glStencilFuncSeparate() to draw only the back stencil setup area by comparing against 0x12. As the front face is culled away, I expect only the areas covered by my back stencil buffer whose value is 0x12 to be displayed.

But to my dismay, it displays blank screen. When it give the value for comparision as 0x5(front stencil init val) with cull face as front, I am able to see the portion of the cube.

So it seems to indicate that, the back face of the cube is compared against the front stencil when culling is set to GL_FRONT. All other parameters like StencilClear(value 0x1) looks proper and i somehow cannot understand this behavior.

#Setup the stencil
                glStencilMaskSeparate(GL_FRONT_AND_BACK,0xff)
				glStencilFuncSeparate(GL_FRONT,GL_ALWAYS, 0x5, 0xff)
				glStencilFuncSeparate(GL_BACK,GL_ALWAYS, 0xc, 0xff)
				glStencilOpSeparate(GL_FRONT,GL_KEEP, GL_KEEP, GL_REPLACE)
				glStencilOpSeparate(GL_BACK,GL_FRONT,GL_KEEP, GL_KEEP, GL_REPLACE)
			    glColorMask (GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE)
			    glDepthMask(GL_FALSE)
                glDrawArrays(GL_TRIANGLES, 0, 6)

#enable culling 
		glCullFace(GL_FRONT)
		glEnable(GL_CULL_FACE)

#Draw a cube
glStencilOpSeparate(GL_FRONT,GL_KEEP,GL_KEEP,GL_REPLACE)
glStencilOpSeparate(GL_BACK,GL_KEEP,GL_KEEP,GL_REPLACE)
glColorMask (GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE)
glDepthMask(GL_FALSE)
glDrawArrays(GL_TRIANGLES, 0, 36)

OpenGL version: OpenGL ES 3.1 Mesa 19.0.8, Mesa DRI Intel Haswell Desktop

Could someone please help me understand why the front and back stencil buffers seems swapped with culling mode?

[GClements]

@dpk

In my code, I have set the front and back stencil buffers separately

You don’t have front and back stencil buffers. There is one stencil buffer; you get to select the behaviour for front and back faces. So the first section of the above code draws two triangles; for each triangle, it writes 5 to the stencil buffer if the triangle is front facing and 12 if it is back facing.

I’m not going to comment on the rest of it because it’s clearly not the actual code you’re using, and I strongly suspect that what you typed isn’t even close to the actual code, in the sense that the differences between the two would render any response based upon the above code irrelevant to the actual code.

[dpk]

Thanks for correcting my understanding on the stencil buffers. I managed to rootcause and fix the issue. Turns out that in the quad i was drawing to setup the stencil buffer, there were only front facing triangles. So the back stencil states were basically having no effects. Adding the back faces to the quad fixed all issues.