Selecting Vetrices


I am trying to write code to allow me to select individual or sweep select many vetrices from a 3ds mesh i loaded in. I have used gluProject to project the model to window coords then compared with the mouse position, however if i say if(mousex == objx && mousey == objy) it never finds a match. If i use just the x value it does, the y value seems to never be matched. Any ideas?

P.S. I also used gluUnProject on the mouse coords and had the same problem. Tell me i am making a silly mistake right? (and i have tried tolerenace of saying if its within x amount of the pixel select it, again this doesn’t work for the y value). It could be something else altogether, how do you select vetrices?

Try replacing your y value with (screen_height - y) and try that…

OpenGL’s Y-axis (used by gluProject) starts in the bottom of the window and goes up, while the window Y-axis (used by the mouse) starts in the top of the window and goes down.

To convert from OpenGL to window Y coordinates, do this:

ogl_y = window_height - 1 - mouse_y

Do i do that before or after i have projected it? And i do it like this right…

GetWindowRect(hWnd, &rect);

realMouseY = (rect.botoom - 1 - mousey);

//Now put the realMouseY value in gluProject

And remember that a float to int typecast may not round the values as desired. Also, comparing two floats is not well done with a float1 == float2. You could use this macro if it helps you:

#define NEPSILON 0.0001f
#define DOUBLE_EQ(x,v) (((v - NEPSILON) < x) && (x <( v + NEPSILON)))

The realMouseY is not used in glProject. You need it afterwards to test wether there is a match or not in coordinates.

sorry, i meant to say use the realMouseY in gluUnProject as i am turning the mouse coord into object coords.

I have given it a try just now and when i rotate my object it selects perfectly but i can only select the objects in distance not facing out towrds me…does that make sense?

What do you mean with not facing out towards you?

ok my model is of a head. So front on you have the eyes, mouth etc. When the head is looking at me, i cannot select the eyes etc…however i just found if i move the mouse above the eyes by a particular offset i can select them when it is facing me.

Now if i rotate the head so that it is facing, the head is looking, away from me into the screen i can select the eyes perfectly (through the mesh).

(when i say facing i mean that if say the head was a real persons head, it is the direction they are looking - really hard to make this clear. )

And what exactly is the code you’re using?
There might be a problem with the z-coordinate of the mouse and the conversion, since your viewport is probably using some fov. Try to get two points for two z-coordinates of the mouse position and then test against the resulting line going through both.

Here is the code i use to match the vetrices, i pass in 1.0 for the mouse z value, as it doesn’t have one. Is that possibly a reason for this problem? (Thanks for all your help so far) —>

bool MatchVertex(GLdouble mouse_x, GLdouble mouse_y, GLdouble tolerance)
GLdouble winx = 0,winy = 0, winz = 0;
GLint viewport[4];
GLdouble modelview[16];
GLdouble projectmatrix[16];


	glGetDoublev(GL_MODELVIEW_MATRIX, modelview);
	glGetDoublev(GL_PROJECTION_MATRIX, projectmatrix);
	glGetIntegerv(GL_VIEWPORT, viewport);

	RECT rect;
			if(!(gluUnProject( mouse_x,
						(rect.bottom - 1 - mouse_y),
					MessageBox(ghWnd,"Error Projecting 2D","GluProject Error",MB_OK | MB_ICONEXCLAMATION);
					return false;

	// Since we know how many objects our model has, go through each of them.
	// Get the current object that we are displaying
	t3DObject *pObject = new t3DObject;

	for(int i = 0; i &lt; g_3DModel.numOfObjects; i++)
	// Make sure we have valid objects just in case. (size() is in the vector class)
	if(g_3DModel.pObject.size() &lt;= 0) break;

	// Get the current object that we are displaying
	pObject = &g_3DModel.pObject[i];
		// Go through all of the faces (polygons) of the object and draw them
		for(int j = 0; j &lt; pObject-&gt;numOfFaces; j++)
			// Go through each corner of the triangle and draw it.
			for(int whichVertex = 0; whichVertex &lt; 3; whichVertex++)
				// Get the index for each point of the face
				int index = pObject-&gt;pFaces[j].vertIndex[whichVertex];

				if((pObject-&gt;pVerts[index].x &gt;= winx - tolerance && 
					pObject-&gt;pVerts[index].x &lt;= winx + tolerance) && 
					(pObject-&gt;pVerts[index].y &gt;= winy - tolerance &&
					pObject-&gt;pVerts[index].y &lt;= winy + tolerance))
					ObjIndex = i;
					return true;


	return false;


If you want to select vertices, I suggest you project the vertices to window space, and not the mouse to object space. Since the third parameter to gluUnProject is 1.0, you get a point that lies on the far clipping plane, probably far away from anything you draw.

I suggest something like this.

1: Get the mouse coordinates.
2: Transform the mouse y coordinate to OpenGL y coordinates using the formula I gave (new_mouse_y = windowheight - 1 - mouse_y).
3: Project each vertex to window coordinates using gluProject.
4: Test for match with mouse_x and new_mouse_y.

It will save you a lot of trouble compared to trying to convert the mouse coordinate to object space. Since the mouse position is a line in 3D space, you need to test all vertices with this line, not a single point along this line.

I believe that the trouble is worth it for a set of many vertices… Simply create the line equation and then take the vertex’ z value and get the line’s x any y values at this z position. That should be far faster than three matrix multiplications per vertex.

in regards to this method:

>Michael:I believe that the trouble is worth >it for a set of many vertices… Simply >create the line equation and then take the >vertex’ z value and get the line’s x any y >values at this z position. That should be >far faster than three matrix >multiplications per vertex.

I don’t fully understand what you are proposing. I should calculate a line from the object vertex to the mouse vertex and see whether it is on the same z as the mouse? sorry i am confused…any chance of some sample code, how i implement the line equation?

No, what i mean is to set the mouse z once to 0 and once to 1 and for each z calculate the the opengl coordinates. Then, you can put a line between these two vertices.

Oh, and since a line between two vertices is defined like this:

x = vert1 + r*(vert2-vert1)
(all but r are vectors here)

x1 = vert11 + r*(vert21-vert11)
x2 = vert12 + r*(vert22-vert12)
x3 = vert13 + r*(vert23-vert13)

r = (x3-vert13)/(vert23-vert13)

So set x3 to the z coordinate of the vertex you want to test to and you’ll get r and you can calculate x1 (x) and x2 (y) with it. Then test x1 and x2 against the corresponding values of the vertex to test to.

Michael, i can kind of see what you are getting at but because of the naming conventions you have used for the variables it has led to confusion.

Could you explain using x, y, and z as names for each so. MouseX, MouseY, MouzeZ, vert1X, vert1Y,vert1Z etc…

Sorry for dragging this out, i do really appreciate the help. Thanks

Micheal, how is that faster? Don’t you still have to test all vertices that respect the equation? The problem for me is speed.

I have seen a CAD program pre-select each entity when you move the mouse. So each mouse move triggers a selection!!
This is very fast because it can be a very heavy CAD model with thousands of entities and there is no noticeable effect on speed.

Now, I suppose there is some optimisation code there. Instead of testing all the vertices of the model, there must be some classification of object’s window coordinates. Or there is some very good selection code!

I figure you can section off your vertices, do a quick test to see which section(s) the mouses line passes through and minimise your testing accordingly. I’m pretty sure they have a name for this technique but I can’t remember it right now. Also is using gluProject or gluUnProject apparently requires you to do 3 matrix multiplications. However, if you use gluUnProject you only do that twice. If you use gluProject you have to do that for all vertices
(ie 3 MatrixMults * 2 vs 3 MatrixMults * Numvertexes)

what Micheal posted can be rewritten

P = N + r*(F - N)
where N is the vector you get when Unprojecting to the near plane (Winz = 0)
F is the point you get when Unprojecting to the far plane (Winz = 1)
P is any point on the line between N and F
r is a scalar with value 0 <= r <= 1

To find the projected point of the mouse at the same depth as each vertex, V, being tested…

r = (Vz - Nz) / (Fz - Nz)
Px = Nx + r*(Fx - Nx)
Py = Ny + r*(Fy - Ny)
Pz = Vz

If the distance between P and V is below a threshold you define, the object can be selected. If more than one vertex is close enough then you have to come up with a scheme to choose. Typically that’s the one on top (with the smallest r) or the vertex that is closest to P

Thanks for all your help everyone, especially Michael and Furrage. The combination of the last 2 posts seems to have solved my problem. When i click the right place is highlighted at last…

	GLdouble r, Px, Py, Pz;
				r = (pObject-&gt;pVerts[index].z - Nwinz) / (Fwinz - Nwinz);
				Px = Nwinx + r*(Fwinx - Nwinx);
				Py = Nwiny + r*(Fwiny - Nwiny);
				Pz =  pObject-&gt;pVerts[index].z;

				if((pObject-&gt;pVerts[index].x &gt;= Px - tolerance && 
					pObject-&gt;pVerts[index].x &lt;= Px + tolerance) && 
					(pObject-&gt;pVerts[index].y &gt;= Py - tolerance &&
					pObject-&gt;pVerts[index].y &lt;= Py + tolerance))			
					ObjIndex = i;
					ind = index;
					return true;

Thats my new code that seems to do the trick for anyone else that look sat this post for the same problem.

Thanks for all your help