MFC OpenGL glRenderMode

MHONDA · April 28, 2021, 4:05am

I have a CAD software I created using MFC/OpenGL. Suddenly, last year, starting with Visual Studio 2019 Community version, it started to crash. I finally figured out why…

During the PICK operation, I was using

numberOfHits = glRenderMode (GL_RENDER)

This was working all along. Now it is crashing the program down the stream.
After struggling with this strange crash for a year, finally, I changed it to

numberOfHits = glRenderMode (GL_SELECT)

Now the numberOfHits is correct and no crash… Wow.

According to MS MFC documentation, GL_RENDER will return “0” which is bad.
I should have used GL_SELECT all along to get the hit count.

But why was it working for the past 10 years with GL_RENDER?

I am confused… It appears like glRenderMode() was altered recently…in MFC OpenGL???

GClements · April 28, 2021, 12:36pm

Unless there’s a bug in the implementation (which is entirely possible; glRenderMode wasn’t a widely-used feature and probably isn’t heavily tested), the return value depends only upon the prior mode and is independent of the mode parameter.

If you call glRenderMode(GL_SELECT), the number of entries added to the select buffer is returned from the next call to glRenderMode (regardless of the mode parameter to that call).

The behaviour isn’t implemented in the OpenGL library (opengl32.dll) but in the device driver for the video hardware. Unless you’re using the fallback software implementation, the OpenGL library simply passes commands to the driver.

MHONDA · April 28, 2021, 3:05pm

Thank you for the reply. Surprise to hear you say that glRenderMode() is not a widely used feature; what do people use to do the pick selection in OpenGL then?

Below is my code. I was using (1) for many decades and it was working OK until I tried Visual Studio 2019 Community version last year. It caused a crash in some OpenGL routine down the stream…
I changed to (2) as shown in this sample code and it fixed the crash…I do not understand why.

(1) hits = glRenderMode (GL_RENDER); // crash in VS2019

(2) hits = glRenderMode (GL_SELECT); // fixed the crash
glRenderMode(GL_RENDER);

/////////////////////////////////////////////////////////////////////////////
bool CGView::PickOperation ( CPoint point ) 
{
	GLuint selectBuf[BUFSIZE];
	GLint hits;
	GLint viewport [4];
	glGetIntegerv (GL_VIEWPORT, viewport); 
	glSelectBuffer (BUFSIZE, selectBuf); 
 	glRenderMode (GL_SELECT); 
	glInitNames();
	glPushName(NON_PICKABLE);  // set no-pick as initial default

	glMatrixMode (GL_PROJECTION);
	glPushMatrix ();  
		glLoadIdentity ();
		gluPickMatrix ((GLfloat)point.x, (GLfloat)(viewport[3]-point.y),10.0f, 10.0f, viewport ); 
		Setup2ProjectionMode ();
		// Force redraw in selection mode
		Invalidate (false);	
		CWnd::UpdateWindow();  // this is needed for pick 

	glMatrixMode (GL_PROJECTION); 
	glPopMatrix ();  
	glMatrixMode (GL_MODELVIEW);   
//------------------------------------------------------------------------------------ FIX
	hits = glRenderMode (GL_SELECT); // 2021-04 changing to GL_SELECT fixed this problem!!!
	glRenderMode(GL_RENDER); // 2021-04  We do need to call with GL_RENDER also here,,,,
//------------------------------------------------------------------------------------ FIX
	bHitResult = ProcessHits ( hits, selectBuf );  
	return bHitResult;
}
/////////////////////////////////////////////////////////////////////////////
bool CGView::ProcessHits ( int hits, GLuint buffer[] )
{
	GLuint noNames, *ptr;
	ptr = (GLuint*) buffer;
	for (int i = 0; i < hits; i++)   // For each hit...
	{   
		noNames = *ptr++; 
		GLfloat minZvalue = (GLfloat)*ptr++;
		GLfloat maxZvalue = (GLfloat)*ptr++;

		for ( int j = 0; j < (int)noNames; j++ )  // For each name...
		{

Alfonse_Reinheart · April 28, 2021, 3:18pm

To the extent that picking is necessary, you just render object indices to an off-screen framebuffer and read back the pixel of interest to find out what’s there.

GClements · April 28, 2021, 4:07pm

If that works, but hits = glRenderMode(GL_RENDER) doesn’t, that looks like a driver bug. Although, it’s possible that you have a memory corruption bug elsewhere and that specific situation triggers it.

On the other hand, this is suspect:

I wouldn’t recommend calling Windows GDI functions while in select mode. Call the drawing code directly.

Either that, or CPU-side ray-object intersection. You don’t have to use the GPU for picking.

MHONDA · April 28, 2021, 4:22pm

Thank you for the reply. I do not have any idea how to do picking using the method you two recommended…
Are you saying the OpenGL pick method I am using is not a recommended way? Well, this is something being described in many OpenGL tutorials for picking…

By the way this is from MS Documentation.

Remarks

The glRenderMode function takes one argument, mode, which can assume one of three predefined values above.

The return value of the glRenderMode function is determined by the render mode at the time glRenderMode is called, rather than by mode. The values returned for the three render modes are as follows.

GClements · April 28, 2021, 5:05pm

Well, it’s about a decade or two out of date. glRenderMode doesn’t exist in the 3.2+ core profile or in any version of OpenGL ES. It’s also something that’s not easy to implement on modern hardware.

It’s the same as the online reference pages and the specification. The return value is determined by the mode you’re leaving, not the mode you’re entering (as determined by the mode parameter). So when terminating select mode, you should get the same value regardless of the mode parameter.

MHONDA · April 28, 2021, 6:06pm

Thank you for the advice. My CAD program is indeed “a decade or two” old.
As I showed you, my Pick selection process is fairly well isolated in my CAD program. Can you point me to some tutorial or examples to do the pick-selection in “the modern way”?