One more stencil question

OpenGL OpenGL: Basic Coding
1

I have a working stencil function with an issue I do not understand.
I have to run the function twice before I can observe the stencil effect in the window.
That has a bad ”cumulative” effect - “maximizing” the window produces a total garbage
I am executing the stencil as a last function in a chain of rendering functions, all in “display”.
The “display / resize ” works as expected without the stencil , no problem “maximizing/ minimizing” etc.

What am I missing ?

PS I can post the stencil function if it helps to resolve this.

2

Guessing somewhat blindly my suggestion would be to check if you are:

  • clearing the stencil buffer (i.e. is GL_STENCIL_BUFFER_BIT included in your call to glClear or similar)
  • clearing the stencil buffer to the value that your stencil logic expects (i.e. correct clear value set with glClearStencil)

If that does not help you can try to run it under renderdoc and compare the stencil buffers for frame 0 and 1.

3

Appreciate the reply, perhaps I should post the code to answer the questions.
It is a work in progress , but working code as originally posted.
I did add
glClearStencil(0x0); without affecting the problem.

What are you calling "frame 0 and 1 ??
It is same as making two passes at the function ???

I noticed I am not disabling the stencil.

int OpenGL_Stencil(int (*Stencil)(int), int parameter = 0) {
	#ifdef TRACK_STATE
	printf("\nTRACKING ENTRY stenmcil  %i function  %s  @line %d ",
			system_state, __FUNCTION__,
			__LINE__);
#endif

// clear all three buffers - not neccessary , default 
	glClear(GL_STENCIL_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
//enable stencil  test
	glEnable(GL_STENCIL_TEST);
// turn off writing to color buffer
	glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
// enable only to verify the stencil
//glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
// setup stencil function
	glStencilFunc(GL_ALWAYS, 1, 1);							
// setup stencil operation
	glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);					//
	glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
// get stencil
// TODO need parameter here to verify stencil 
	parameter = Stencil(parameter);				// TOK parameter);

	glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
//
	// switch keep area
	if (parameter == 0) {
		printf("\nStencil full circle parameter %i @function %s @line %i",
				parameter, __FUNCTION__, __LINE__);
		glStencilFunc(GL_EQUAL, 1, 1);
	}	// keep inside circle
	else {
		printf("\nStencil last circle parameter %i @function %s @line %i",
				parameter, __FUNCTION__, __LINE__);
		glStencilFunc(GL_NOTEQUAL, 1, 1);
	}	// keep all outside of stencil
//
	glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
	glutSwapBuffers();
// BUG does not update until passive mouse move 
	printf(" \nEXIT @function %s @line %i @time %s ", __FUNCTION__, __LINE__,__TIME__);

#ifdef TRACK_STATE
	printf(
			"\nTRACKING EXIT  stencil  %i function  %s  @line %d ",
			system_state, __FUNCTION__,
			__LINE__);
#endif

	return parameter;

}
4

In my experience the usual structure for many OpenGL (or rather graphics related in general) programs is to have a big loop that handles events and performs all the rendering and ends with swapping buffers. The rendering output from one iteration of this loop is often referred to as a “frame”. I thought that is what you were referring to when you said you get the expected result when your function is called more than once - I see now that was probably a misunderstanding on my end.

You are calling glutSwapBuffers, so I assume you are using GLUT for your program. In a GLUT based application the loop is not directly visible because it is inside the GLUT library (in glutMainLoop) and it just calls out to the various functions you’ve registered with GLUT. Most of those functions deal with user input (or similar things caused by user interaction, like window resizing), but the display function (registered with glutDisplayFunc) is the part of the loop body that does all the rendering.
With that in mind it is very odd/unusual/surprising to me to see no rendering in your posted function (without rendering anything the only operation that modifies values in the stencil buffer is clearing it) and that it ends with glutSwapBuffers. There should normally be only one and exactly one call to glutSwapBuffers at the end of your display function. Of course it can be in a function called from the display function, but then that function should only have a single call site near the end of the display function.

You are also calling glStencilOp repeatedly without rendering anything in between, which makes all but the last call have no effect other than wasting a little bit of processing time. OpenGL is a state machine, most commands manipulate that state and all of that state acts as implicit input to commands that actually render (e.g. the glDraw* commands). Unless you render something after setting a state it does not affect the image you get.

I’m afraid I don’t have any more specific suggestions what is wrong. You did not describe what it is you actually want to do and I cannot guess from the code you posted - the things I described above just make it too “unusual” for me to figure out.

PS: When posting code please make sure it is reasonably formatted, i.e. consistent indention, dead code removed, and ideally has comments that are meaningful to others. After all you are hoping others will put effort into answering your question, show that you put effort into asking it :wink: - see the Forum Posting Guidelines for more suggestions on how to maximize the chances of getting a helpful answer.

1 Like
5

I was afraid that my "work in progress " code gets criticized one way or another.
I also have a small issue when people do not understand the code and by some miracle they will gain understanding if they know what I am trying to accomplish.

So just to repeat - the stencil works when accessed twice.
When accessed THIRD time the view is just garbage.

So you must be correct - there is something missing in rendering.

It has nothing to do with matrixview, as I thought, it fails after first object to be stenciled is rendered. I have also tried display / reshape with same results.

I guess I’ll keep debugging it until I fix it.

In closing, I still appreciate all your comments.
It helps to have a second opinion.

6

FYI
I am sure I found “THE BUG”.
I have build and test the object to be stenciled first , then applied the stencil.
WRONG!
I have the do stencil first , sort of wrapper over the objects. Easy, just some code reshuffling.