The first thing you need to do is generate a ray from the mouse cursor position into the scene. This is done by unprojecting the mouse position. This is a function from my most recent project that does a similar thing. Note that the mistake most often made is to get the ray with the incorrect matrix on the stack. Be sure to execute this function after executing your current camera modelview and projection matrix and not a local matrix for any scene objects.
//
// Get the ray from the cursor into the scene - no selection occurs - this is just a utility method.
//
void CSelectionBuffer :: GetRay ( int MouseX, int MouseY, CVertex3D& p1, CVector3D& p2, float z )
{
//
// Finally, we need to reverse transform the values we have back into the world to get correct world x, y, z.
//
GLdouble Projection [16], Model [16];
GLint Viewport [4];
//
// Now get the matrices and viewport.
//
glGetDoublev ( GL_PROJECTION_MATRIX, Projection );
glGetDoublev ( GL_MODELVIEW_MATRIX, Model );
glGetIntegerv( GL_VIEWPORT, Viewport );
//
// We will need to invert mouse y (because its top left whereas our window is bottom left origin).
//
double Invert_m_y = static_cast<double> ( Viewport [3] ) - static_cast<double> ( MouseY );
HITRECORD h1, h2;
//
// Inverse transform the given point - note its important that the viewing transforms are setup as they were rendered.
//
gluUnProject ( static_cast<double> ( MouseX ), // Mouse x.
Invert_m_y, // Inverted mouse y.
0.0, // The z depth result we got from selection.
Model, // Model view matrix.
Projection, // Projection matrix.
Viewport, // Viewport co-ordinates.
&h1.x, // Final result x.
&h1.y, // Final result y.
&h1.z ); // Final result z.
gluUnProject ( static_cast<double> ( MouseX ), // Mouse x.
Invert_m_y, // Inverted mouse y.
1.0, // The z depth result we got from selection.
Model, // Model view matrix.
Projection, // Projection matrix.
Viewport, // Viewport co-ordinates.
&h2.x, // Final result x.
&h2.y, // Final result y.
&h2.z ); // Final result z.
//
// Copy results across ( from double to float ).
//
p1.v [0] = static_cast<float> ( h1.x );
p1.v [1] = static_cast<float> ( h1.y );
p1.v [2] = static_cast<float> ( h1.z );
p2.v [0] = static_cast<float> ( h2.x );
p2.v [1] = static_cast<float> ( h2.y );
p2.v [2] = static_cast<float> ( h2.z );
//
// Turn inverse transformed vertex into a unit vector.
//
if ( ( p2 - p1 ) == 0 )
{
return;
}
else
{
p2 = ( p2 - p1 ).Normalized ();
}
}
Next you need to find the intersection of an infinite ray and a triangle (lots of tuts on the net all over, search on ray riangle intersection) - you will need to check all intersections and use the intersection thats closest to you(distance from intersection point to camera eyepoint).
Thats more or less it I guess.