Whats wrong with the picking?

Hi all:
im coding a project using Glut, and im having a problem with my picking/rendering procedures:
[ol][li] the project consists of three sub windows…[] in one of them im drawing a menu with buttons[/li] and shapes.
(not using any graphic library at all)[li] now, all buttons and shapes are objects which[/li] have there own drawind/rendering methods.
(again, MY objects)[li] im having troubles with the “naming thing”…[
] every object (a button or a shape) has its own _ID assigned to it at the main entry of the program… that _ID is used as the “name” for the selection mode.[*] Now here is my menu drawing procedure:[/li]

void drawMenu(GLenum mode)
{
glMatrixMode(GL_MODELVIEW);
glPushMatrix();
		
glOrtho(0.0,MAINWINX*WINRATIO,0.0,MAINWINY*WINRATIO,100.0,-100.0);
		
renderBackground();	//.................... Render the background area

if(mode==GL_SELECT)
{
  glInitNames();
  glPushName(0);
}

 glTranslatef(30.0,5.0,0.0);
 for(int btn=0; btn<MenuObjectsCounter; btn++)
   if(Menu[btn])
      Menu[btn]->Render(mode);
					
  glTranslatef(-30.0,-5.0,0.0);
  glPopMatrix();
  glFlush();
  glutSwapBuffers();
}

[li] every Render() procedure called with the current ‘mode’… so the objects can choose its own ‘Draw()’ method (either ‘pressed’ or ‘normal’.[*] Now the mouse call back function is as follows:[/li]

void pickMenu(int button,int state,int x,int y)
{
	GLuint selectBuf[BUFSIZE];
	GLint hits;
	GLint viewport[4];
		
	if( button==GLUT_LEFT_BUTTON && state==GLUT_DOWN)
	displaySubWin3();// the menu window

	glGetIntegerv(GL_VIEWPORT,viewport);
	glSelectBuffer(BUFSIZE,selectBuf);
	glRenderMode(GL_SELECT);

	glMatrixMode(GL_PROJECTION);
	glPushMatrix();
	glLoadIdentity();
	gluPickMatrix( (GLdouble)x , (GLdouble)(viewport[3]-y) , 2.0 , 2.0, viewport );

	glOrtho(0.0,MAINWINX*WINRATIO,0.0,MAINWINY*WINRATIO,100.0,-100.0);

	drawMenu(GL_SELECT);

	glMatrixMode(GL_PROJECTION);
	glPopMatrix();
	glFlush();
	glutSwapBuffers();

	hits = glRenderMode(GL_RENDER);
	processHits(hits,selectBuf);
}

[li] HERE IS MY PROBLEM: I figured that no matter what i do ‘hits’ value is always ‘0’!!![/li]so i can’t process the hits!!![/ol]

The problem is that you are pushing the same name each time…

glPushName( 0 );

whereas you should be calling

glPushName( btn );

for each separate object where btn is your index in the for loop and should therefore be part of your button rendering loop. Be sure to call

glPopName();

before the next glPushName or you will get more than 1 name returned for each hit

0
0 1
0 1 2
0 1 2 3

as opposed to

0
1
2
3

Hope that helps.

STILL, SAME PROBLEM.
maby if i simplify the problem the issue will be more understood…
suppose i have no buttons no nothig…
in the menu window im drawing a simple circle and nothing more…
here is the code for that:

void drawMenu(GLenum mode)
{
	BCircle c1(50,50,120,15,0.0,1.0,0.5);
	BCircle c2(50,50,120,15,1.0,0.0,0.5);
	
//	glMatrixMode(GL_MODELVIEW);
	glPushMatrix();
	glOrtho(0.0,MAINWINX*WINRATIO,0.0,MAINWINY*WINRATIO,100.0,-100.0);
	renderBackground();	//.................... Render the background area

	if( mode == GL_RENDER )
	{
		if(clicked)
		{
			glTranslatef(20,20,0);
			c2.Render();
		}
		else
		{
			glTranslatef(20,20,0);
			c1.Render();
		}
		
	}
	if(mode==GL_SELECT)
	{
		glInitNames();
		glPushName(0);
		if(clicked)
		{
			glLoadName(2);
			glTranslatef(20,20,0);
			c2.Render();
		}
		else
		{
			glLoadName(1);
			glTranslatef(20,20,0);
			c1.Render();
	
		}
		
	}
	glPopMatrix();
	
	glFlush();
	glutSwapBuffers();
}

and lets say the Mouse Function is:

void pickMenu(int button,int state,int x,int y)
{
	GLuint selectBuf[BUFSIZE];
	GLint hits;
	GLint viewport[4];

	
	fprintf(MyLog,"
_________________>>>Entering pickMenu()");
	
	 if( button==GLUT_LEFT_BUTTON && state==GLUT_UP)
		displaySubWin3();

	drawRegion(x,y);
	glGetIntegerv(GL_VIEWPORT,viewport);
	glSelectBuffer(BUFSIZE,selectBuf);
	glRenderMode(GL_SELECT);

	glMatrixMode(GL_PROJECTION);
	glPushMatrix();
	glLoadIdentity();
	gluPickMatrix( (GLdouble)x , (GLdouble)(viewport[3]-y) , 2.0 , 2.0, viewport );

	glOrtho(0.0,MAINWINX*WINRATIO,0.0,MAINWINY*WINRATIO,100.0,-100.0);

	drawMenu(GL_SELECT);

	glMatrixMode(GL_PROJECTION);
	glPopMatrix();
	glFlush();
	glutSwapBuffers();

	hits = glRenderMode(GL_RENDER);
	
	fprintf(MyLog,"
_________________>>> %d objects were picked",hits);
	
	processHits(hits,selectBuf);

}

and here is my hits process func:

void processHits(GLint hits,GLuint selectBuf[])
{
	int i;
	unsigned int times=0;

	fprintf(MyLog,"
.........................processing %d hits",hits);

	for(i=0;i<hits;i++)
	{
		if(selectBuf[i]==2)
		{
			fprintf(MyLog,"
.............. circle was picked %d times",times);
			clicked = 1; // GLOBAL VAR

		}
		else
			clicked=0;
	}

}

the problem is that i get zero hits…
(that i know from MyLog file…)
Please, HELP!!!

I think your matrix states have been messed up. My guess is you are drawing the menu at a different place when you are in select mode and in render mode.

In your pickMenu function, you switch to projection mode, push, and set your matrix to glOrtho*gluPickMatrix, when you call drawMenu, you push again and this time set to glOrtho only(no gluPickMatrix). And you also call all your rendering routines in Projection mode while in select mode(switch to ModelView is commented out) which might further mess up thinks depending on the behaviour of renderBackground(), c?.Render() functions.

As a suggestion remove the matrix mode state setups from the drawMenu function, as you are doing it already in your pickMenu function. Do it the same way when you are actually drawing the menu.

As a test you can convert you pickMenu function in to a menu rendering function. just remove glRendermode function (and possibly gluPickMatrix too) and see where your menu gets rendered.

obirsoy,
first of all thanks for your reply,
i tried to do what you suggested, although this is not it… i removed all the matrices that might deform the drawings, and as you sayed,i treated pickMenu as a drawing function… the menu was drawn as it is…
However, i wanted to know whether the use of the Double Buffer might affect the picking or not!!
because my display mode is actually double buffered… as if i draw to the frozen screen in the back buffer and pick from the front buffer…
I don’t know about that but if that is my problem… i would like to know…

THANKS ALL

p.s.
Hey!
Maby this will help…
Here is my menu drawing code…
See, when i push, load, whatever to, from the select buffer, it just does’t fill up!!
I did some printing to the screen, and it just printed Buffer[i]=0

void drawMenu(GLenum mode)
{
	if(mode==GL_RENDER)
		printf("Entering display procedure in RENDER mode
");
	else
		printf("Entering display procedure in SELECT mode
");
		
	glMatrixMode(GL_MODELVIEW);
	glPushMatrix();
		
		glOrtho(0.0,MAINWINX*WINRATIO,0.0,MAINWINY*WINRATIO,100.0,-100.0);
		
			renderBackground();	//.................... Render the background area

		if(mode==GL_SELECT)
		{
			glInitNames();
			glPushName(0);
		}

	 	glTranslatef(30.0,5.0,0.0);
	
			for(int btn=0; btn<MenuObjectsCounter; btn++)
			{
				if(Menu[btn])
				 	Menu[btn]->Render(mode);
				
				if(mode==GL_SELECT)
				{
					glLoadName(btn);
					glPushName(0);
					cout<<"selectBuf["<<btn<<"] = "<<selectBuf[btn]<<endl;
				}
			}
					
			
			glPushMatrix();
				glTranslatef(71+13,30+26,0.0);
				RightTrngl->Render(GL_RENDER);
			glPopMatrix();
		
			glPushMatrix();
				glTranslatef(30-13,30+26,0.0);
				glRotatef(180,0.0,1.0,0.0);
				LeftTrngl->Render(GL_RENDER);
			glPopMatrix();

		 glTranslatef(-30.0,-5.0,0.0);

		glPopMatrix();
}

This will drive me crazy, im telling you!!

Friends, if some one realy wants to help, please post your e-mail here, i will send the project (Visual C++) as soon as i get it…
(it might be a tiny problem…how knows!)

You should also NOT cal swap buffers while in GL_SELECT mode. Verify that your mouse coordinates are in the same frame of reference as your OpenGL frame of reference (i.e. OpenGL has x=0 at left and y=0 at bottom - some systems have x=0 at left and y=0 at top). Also take the glOrtho call out of drawMenu function as it is redundant and will probably result in a matrix skewing what you expect.

OK shinpaughp,
that was it!
I had to change the returned mouse coordinates in the picking matrix. The reason for that is exactly what you sayed…

int h = glutGet(GLUT_WINDOW_HEIGHT);
...
gluPickMatrix( (GLdouble)x , (GLdouble)h-y , 1.0 , 1.0, viewport );

My window system is Windows thats why i had the y coordinate flipped!
Any way thanks…it realy helped!!!