OpenGL stencil analysis #2

Please just verify.

  1. I have read somewhere that glStencilMask which controls the data written to stencil (buffer) is enabled by default. It does not seems to be true. Yes / NO
  2. Since stencil is done “per bit / pixel” and the GL_TRUE is single bit does this only enables the mask and if so WHERE is such mask set?
  3. Same for glColorMask.
  4. Again per some tutorial - the glColorMask enables / disables writing to color buffer. It was presented as ways to just verify the stencil. However, with glColorMask DISABLED I get no data in stencil buffer.

What am I doing wrong ?
DO I have to do more to write data to stencil buffer?
Do I need "double buffering "?
With glColorMask ENABLED I get BOTH stencil and correctly rendered fragment
to display! .

	glStencilMask(GL_TRUE); // anable writing to stencil buffer (??)
		// turn off writing to color buffer
		// enable only to verify the stencil
		//glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);

glStencilMask isn’t something which is enabled or disabled. The initial value is all-ones, i.e. all stencil bits can be modified. Setting some mask bits to zero results in the corresponding bits being unchanged by any stencil operation.

What are you talking about? The argument to glStencilMask is a GLuint, a 32-bit unsigned integer.

glColorMask takes four GLboolean values, one for each component (red, green, blue, alpha). It isn’t possible to mask individual bits when writing to the colour buffer, only entire components. Similarly, glDepthMask enables or disables writes to the depth buffer entirely.

To enable or disable writing to the stencil buffer in its entirety, use glStencilMask(~0U) or glStencilMask(0U) respectively.

Hard to say without knowing what you’re trying to do.

Double-buffering only affects the colour buffer.

OK, if stencil mask is by default set to all 1’s. there is no need to manipulate it unless SPECIFIC stencil bits need to be controlled.
My task is very simple - I have an objects constructed, copied and rotated / translated/ scaled from a single ( main unity) circle.
Parts of these objects are outside of this main circle ( by rotating the main object ) so I want to apply unity circle stencil to extract ONLY the inside of the main and subsequent objects.

glStencilFunc(GL_ALWAYS, 1, ~0U);
// draw filled circle
glStencilFunc(GL_GREATER, 0, ~0U);
// draw other stuff

This is essentially what I am doing.
I went back to my red book and loaded stecil.c sample.
It does not display anything. I have a choice to find out why or continue to work on my project.
Few interesting things I noticed in stenci.c.
“They” enable stencil test early in main event loop.
I have not look if it is ever disabled.

The stencil setup is done in “display” and so is mine, but my is buried under few layers of functions.
Most interesting part is - the actual stencil , where the values are (expected to be 1 ) masked with (default) “1” is preceded with
glStencilFunc(GL_EQUAL, 1,1);

It to me looks as the stencil is double defined and immediately tested.

Here is a real, partially working pseudo code description
In an essence
enable stencil
clear buffers
do not enable / disable color mask - ! work just OK without
do stencil function / stencil operation - not sure why
render stencil - red unity circle
do stencil function - to select INSIDE of unity circle
do stencil operation - this is where I suspect my error is
render object to be stenciled
enable color buffer - not sure why
disable stencil

The result is
stenciled object is WITHIN the unity circle - TOK
so is the original red unity circle stencil.

I do not know HOW to get rid of it !

And I am sorry to be such a…

			//glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);  // onserve object
			glStencilFunc(GL_ALWAYS, 1, ~0U);
			glColor3f(1.0f, 0.0f, 0.0f);
			//return 0;
			//glStencilFunc(GL_ALWAYS, 1, 1);
			//glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
			glStencilFunc(GL_EQUAL, 1, 1);
			OpenGL_Render_RealAxis(); // int parameter = 0)



'// set stencil "color" !!! 
**glColor3f( 0.0f, 0.0f, 0.0f);**

What a hack !

This did not in fact solve your problem. It simply set the per-vertex color of the “unity circle”. And odds are good that you’re just setting it to the background color.

Your problem persists; you’re “unity circle” still has a color. That the color happens to be the background color is irrelevant; if you change your background color, the problem will re-emerge. You don’t want to write the color of the “unity circle”. That’s why GClements’s code disables color writes during the “draw filled circle”, which your version didn’t do.

You didn’t solve your problem; you just masked it out. It’s like painting over a small hole in the wall; the hole is there, you just can’t see it.

Agree, but I needed the color to do more debugging.

Also setting
glStencilFunc(GL_ALWAYS, 1, ~0U);

to " always " and me saying the parameters are ignored is not totaly  correct. 
They are still used by following  stencil operation function.

But for now I need to clean it all up and keep the background color as is.

One more… I hope last one about stencil.

There are MANY pictures and text documents about OpenGL “pipe rendering”
Here is one

There are NONE such documents describing sequence of stenciling operation.
Don’t get me wrong - there are bits of info mostly describing relations between TWO adjacent functions.

For examples
one needs to select / set the “reset value” BEFORE “clearing” stencil buffer.

Generic OpenGL states that " until stencil test is enabled" operations do not have effect on the stencil buffer. ( In red book stencil test is enabled in main event loop.)

Basically - it would be nice to have similar documentation on stencil coding flow as currently exists on OpenGL rendering pipe description .

Mrs Google does not know of any such thing,

That’s because the “sequence of stencil testing” isn’t a “sequence”. It’s just one operation: the test.

Now yes, to actually perform an effect using the stencil buffer, you generally need two separate rendering operations: one which lays down certain stencil values, and another that tests the fragment’s reference stencil against those values. But that’s a matter of achieving a particular effect, not a matter of what the feature is.

Descriptions of the rendering pipeline explain how the various stages of rendering work. They don’t explain what you do with them; that is, the effects you can achieve through them.

That is, these are subjects for tutorials, not reference/descriptive documentation. And there are several tutorials that show the details of stencil effects.

… I just did a search for OpenGL tutorials that talk about stencil operations. The first two I found (linked above)… never actually mention glClearStencil. They use the default clear stencil value of 0. I mean sure, it’s the default, so if you never set it, and its the value you want, then it works. But I find myself disappointed in the OpenGL ecosystem for not bringing up that function.

Granted, because you’re using zero, it doesn’t actually matter in your case either, so if you never knew about the function, your code would still work :wink:

Yes, I have been using one of the references.
It is very misleading and parts are missing (GL_ALWAYS).
It skims over setting stencil mask…
I do not mind missing stuff, but find most of the stencil applications / code on net are just copies of each other.
I am always appalled how some “authors” with minimal or no teaching abilities are elevated to experts status simply by "quoting " others of similar qualifications.
Of course there always will be the ones who will shout “what do you expect for free ?”.

"That’s because the “sequence of stencil testing” isn’t a “sequence”. It’s just one operation".

I woudl have to challenge that statement.
Computers by design follow sequence of instructions and such sequence indeed does matter in stenciling.

It has been already mentioned that the “clear” value should be set prior to “clearing” the buffer - hence a definite and well defined sequence.
From such obvious and simple to ridiculous - why would you wanted to use stencil mask BEFORE you define it?

If the red book example enables stencil test (in main event loop ) then default stencil mask must be active. So does each rendering (of anything) fills BOTH color and stencil buffer ? ( If clearing buffers is time consummating , per doc, why wasting time ALWAYS writing to stencil buffer ?)

That is what I am looking for.


That’s interesting; got any evidence to back it up? I mean, I’m totally willing to believe that 2nd and 3rd tier tutorials and learning materials are just copy-and-paste jobs of other, more prominent, materials. But if you’re claiming that most obviously decent ones are, I’m going to need some evidence for that.

For example, the two tutorials I linked to are not copy-and-pasted from each other.

Who is doing the “elevation” in this scenario? Maybe you’re confusing “here’s a link to some learning materials” with “this person is a super-qualified OpenGL expert so you’d better listen to them”.

I would say “what do you expect from a Wikibook?” It’s been my experience that Wikibooks are all, to some degree, heaping dumpsterfires.

There is a difference between understanding how to do a basic operation and understanding how to use that operation to produce some effect.

Adding two numbers is a basic operation. Subtracting two numbers is a basic operation. So is multiplication.

Using these tools to solve for x in the equation Ax + B = 0 is not a basic operation. All the tools are there, but you have to figure out how best to use them to achieve your desired goal.

Reference documentation explains how tools work; in my analogy, they explain how addition and subtraction work. They don’t explain how to apply them to solve a problem.

Figuring that out is your job as a programmer. And expecting reference documentation to tell you how to apply the tools is like expecting a hammer to come with instructions for how to build a house.

The reference documentation for glClear will tell you that it uses the current stencil clear value if you use the stencil buffer bit. It is on you to realize that this means that, if you want to clear a stencil buffer to a value which is not the previous clear value (or the default if you never set it), you need to modify this clear value before calling this function. That is how reference documentation works. And if you ever want to get anywhere as a programmer, you need to get used to that.

The idea with a tutorial is to teach you how a tool works by showing you an application of that tool. But by that very nature, most tutorials won’t be able to completely explain how a tool works. A single application of most tools of any complexity can’t show you everything about how they work.

It is not the job of this tutorial to teach you everything there is to know about the stencil test/op tool. Its job is to show you one way to use it, and thereby help you understand how it works in general.

Learning how to use a tool from a tutorial is like learning how to use a circular saw by watching someone make a table. Yes, you’ll get the gist of it, but you may not fully appreciate all of the other ways to use that circular saw. At the end of the day, complete understanding of a system can only be gained from more direct reference materials.

And that requires being willing to sit down and put the work into understanding the system yourself.

That depends on the stencil operation values.

The default value for GL_STENCIL_PASS_DEPTH_PASS (ie: what happens if both the depth test and stencil test pass) is GL_KEEP. Therefore by default, the stencil value in the stencil buffer is not modified by a fragment that passes the stencil test (a disabled stencil test counts as passing).

Well, I am not sure the rest of the participants appreciate you and I discussion.
But I do definitely appreciate all of your comments.
I especially appreciate the comment on “equation” etc.

After struggling with stenciling I have this analogy to illustrate my concerns about sequencing the stenciling code.

In an essence , stencil is application of logical AND:
Stencil AND Object = Result

Normally the “AND” “sequence” can be implemented A AND or B AND A.

Applying same “AND” principle
“Stencil AND Object” does not gives same result as "Object AND Stencil "

That’s all for today. Got to run…

And that right there is 100% of your problem: you’re seeing “stencil” as an application, not a tool.

Stencil test performs a logical operation. What an “Object” is, what “Stencil” means and what “Result” you desire are entirely up to you.

You’re looking at a circular saw and trying to understand it only in the context of a thing that makes tables.