OpenGL stencil analysis #3 - working on stencil result

Assumptions / known and unknown(?) :

  1. OpenGL stenciling process is a logical operation between stencil(object) and objects to be “stenciled”.
  2. The stencil(object) is build in stencil buffer
  3. The object to be stenciled are build in (?) buffers ? ( or does it matter ?)

The above stenciling process works as expected.

The unknown is
Where , in which buffer/ matrix etc. , is the result of stenciling process?
How do I access it ?

Reason for asking is -
it is desired to resize and move the RESULT of the stencil process, however,
using glScale resizes the original stencil ( object ) NOT the stenciling result.
Same with glTranslate or glWievport. They all WORK on original stencil object , not on stenciling result.

Stencilling works on fragments, not objects. When GL_STENCIL_TEST is enabled, each rendered fragment is tested by comparing the fragment’s current stencil value to the reference value according to glStencilFunc; if it fails, the depth and colour buffers are unmodified. Regardless of whether stencil testing is enabled, each rendered fragment modifies the fragment’s stencil value according to glStencilOp. Both operations require that the framebuffer has a stencil buffer.

The framebuffer. glEnable(GL_STENCIL_TEST) and glStencilFunc affects what ends up in the colour, depth and stencil buffers, glStencilOp affects what ends up in the stencil buffer.

Typically, you don’t. Rendering operations modify the stencil buffer which is used to modify subsequent rendering operations. If you need to read its contents for some reason, use glReadPixels with a format of GL_STENCIL_INDEX.

OK, what I call “result” , your “fragment”, is therefore in stencil buffer. NOT in color buffer, right ?
So the “subsequent rendering operations” do not work as “normal” color buffer, but they are applied to stencil buffer.
That is why I cannot reshape the “result” only, but reshaping the entire stencil.
That is NOT the operation I need.
As illogical as it sound now - I need to select the final stencil area BEFORE I apply the stencil. Or find a different way to accomplish this without stencil.
In other words - doing “zoom” on specific part of the object and moving , resizing such part later in not normal stencil OpenGL ways to accomplish such task.
I’ll try to render / show the “zoom” object and then render it again resizing and translating so it does appears as "centered and magnified " on screen - without stenciling.

It looks as my problem is me getting too "smart / fancy " using OpenGL.
My “fragments” are build from multiple parts, or fragments, each on of them in its own matrix - using push / pop. I still do not understand how stencil works on ALL of them but I cannot control then - select only part of them.
I am rebuilding them and it is slow going because they are no longer independent, but in single pipe. .
Since I am using multiple , cascaded scaling and translation I have to "re-scale and re-translate " each step back to original.
Kinda off defeats the sub-functions and push /pop concept I was getting used to.

No, your principle problem is that you don’t seem to understand what people are saying. For example:

That statement doesn’t make sense. A “fragment” is not a thing that can be built “from multiple parts”. And it definitely doesn’t have “its own matrix” (not in compatibility GL 1.1).

Primitives are rasterized into pixel/sample-sized chunks called “fragments”. Each fragment undergoes some processing, which may or may not result in its values being written to a pixel (corresponding to the fragment’s location in window-space) of one or more of the various buffers in the current framebuffer. If a fragment does have its values written to one or more buffers, those buffers have no idea where that data came from; it’s just data in the buffer.


No, what you call “result” is not a “fragment” at all.

These are not pedantic trivialities; this is the difference between 2 + 2 = 4 and 2 + 2 = Love.

In order to truly understand a system, you have to be able to think in the terms that this system uses. And that means correctly using terminology for that system. Failing to do this accounts for about 80% of the various problems you’ve described over the last month or so. You keep fighting against OpenGL, trying to fit it into your own mental framework.

You need to approach OpenGL on its terms, adapt your thinking to that of OpenGL.

There is no “result”. There is no “zoom”. There is no “object” or “moving” or whatever. Start from the smallest pieces of whatever it is you’re trying to achieve and understand that fully. Then move to a different piece.

Here is a “normal people” definition of stencil

a thin sheet of cardboard, plastic, or metal with a pattern or letters cut out of it, used to produce the cut design on the surface below by the application of ink or paint through the holes.

Adapting such definition - stenciling process produces a product.
Accepting the OpenGL terminology - stenciling process product is “a fragment”.

If I want to further process such product / fragment - where does it reside?

Using similar analogy with “normal” definition - I do not desire to CHANGE the stencil, I want to change the “fragment”. Perhaps applying another stencil to it.

So we’re moving forward with the vernacular terms instead of the technical terms. That’s fine.

No, you can’t do that. You can’t mix vernacular terminology with technical jargon. If you want to speak in vernacular terminology, that’s fine, but you can’t throw technical terms in there and expect the result to make sense.

And what you just said does not make sense.

In the vernacular, a “stenciling process” results in an image. A fragment is not an image. Indeed, the OpenGL concept “fragment” has no useful mapping to your vernacular thinking.

If you want to understand “stenciling” in the vernacular sense in terms of OpenGL work, then the mapping is as follows. Note: when I’m using vernacular terminology, I will use “quotes”; when I’m using OpenGL terms, I won’t.

The “thin sheet of cardboard, plastic, or metal with a pattern or letters cut out of it”? That is essentially what resides in the stencil buffer (which is part of the framebuffer). At each pixel in the framebuffer is a stencil value. You can use that value to decide whether that pixel is considered “cut out” or not.

To “cut” the pattern into the “stencil”, you render a shape to the framebuffer. During that rendering, you turn stencil writes, and you write some value which is different from the values already in the stencil buffer (probably what you cleared the stencil buffer to). The parts of the framebuffer whose stencil value is the value you selected represents the “cut” parts of the “stencil”; the values which are unchanged are the non-“cut” parts of the “stencil”.

The “application of ink or paint through the holes” part is the second phase of the rendering process. Having produced the “pattern”, you now render the stuff you want to “cut” with your “stencil”. To do this, you need to cull out any parts of this rendering where the stencil value in the framebuffer underneath the object being rendered is not one of the “hole” values written in the first part of the process.

Rendering a primitive means to break the primitive up into little pixel-sized pieces called “fragments”. Each fragment is aimed at a specific part of the framebuffer. So there is a stencil value “underneath” it, that is currently sitting in a pixel in the stencil buffer that this fragment is about to be written to.

The stencil test can be used to check to see if the stencil value underneath each fragment matches the “cut” stencil values you wrote in the first part. If it is the “cut” value, then you write that fragment’s color to that pixel in the framebuffer; otherwise, the fragment writes nothing. But you also don’t want to write new stencil values, as that would represent changing the “cut” areas of your “stencil” pattern. So you have to turn off writing to the stencil buffer when doing this.

1 Like

Where , in which buffer/ matrix etc. , is the result of stenciling process?

Let me start again.
There are TWO buffers (stencil and color) involved in stenciling process.
To this day I am not sure WHERE , in which buffer, is the result.

Is is possible to save / copy the result of the stenciling process ?

Select from bellow

I like to use OpenGL ability to build / copy buffers, mainly to be able to render the stencil result from “memory” instead of re rendering it each time I need it.

Yes. The efficient way to do this is using framebuffer objects (so that everything stays in GPU memory). The inefficient (but compatible with OpenGL 1.x) approach is to use glReadPixels to copy the stencil buffer contents to system memory then glDrawPixels to copy it back.

I am looking at this

and you have given me a good start.
Appreciate that.