# Mouse picking with OpenGL

Hello, I know this may not be an OpenGL question, though I assume several users on this forum did mouse picking in their graphics applications before and I was hoping I could get some help regarding the issue I am having with my 3D mouse picking.

I have a basic scene and I am trying to pick a mesh (a triangle in my case) and move it around with my mouse. So far I have been able to implement that correctly but I can’t seem to be able to pick the object from far away. I’d have to move the camera closer to the triangle in order for it to intersect with my ray. I will include code that is relevant to the mechanic below in case I’m doing something wrong with my calculation, thank you for reading my thread. EDIT: Essentially what I’m trying to do is simply make it so that you can pick the object up and move it around from a reasonable far distance, instead of having to move really close to it first in order to pick it up.

Get the ray direction

``````
// Function that takes mouse position on screen and return ray in world coords
glm::vec3 PhysicsEngine::GetRayFromMouse()
{
glm::vec2 ray_nds = glm::vec2(mouseX, mouseY);
glm::vec4 ray_clip = glm::vec4(ray_nds.x, ray_nds.y, -1.0f, 1.0f);
glm::mat4 invProjMat = glm::inverse(m_Camera.GetProjectionMatrix());
glm::vec4 eyeCoords = invProjMat * ray_clip;
eyeCoords = glm::vec4(eyeCoords.x, eyeCoords.y, -1.0f, 0.0f);
glm::mat4 invViewMat = glm::inverse(m_Camera.ViewMatrix());
glm::vec4 rayWorld = invViewMat * eyeCoords;
glm::vec3 rayDirection = glm::normalize(glm::vec3(rayWorld));

return rayDirection;
}
``````

Check for ray-sphere collision

``````
// Function that checks for ray-sphere intersection and returns true or false
bool PhysicsEngine::ray_sphere(vec3 ray_origin_wor, vec3 ray_direction_wor, float sphere_radius)
{
vec3 v = glm::vec3(m_Transformation.GetPos().x, m_Transformation.GetPos().y, 0.5f) - m_Camera.GetCameraPosition();
float a = glm::dot(ray_direction_wor, ray_direction_wor);
float b = 2 * glm::dot(v, ray_direction_wor);
float b_squared_minus_4ac = b * b - 4 * a * c;

if (b_squared_minus_4ac > 0)
{
float x1 = (-b - sqrt(b_squared_minus_4ac)) / 2.0f;
float x2 = (-b + sqrt(b_squared_minus_4ac)) / 2.0f;
if (x1 >= 0.0f && x2 >= 0.0f)
return true;
if (x1 < 0.0f && x2 >= 0.0f)
return true;
}

return false;
}
``````

Utilizing the mechanic

``````
case SDL_MOUSEMOTION:
{
// Check if user is not picking the triangle
if (!bReplacingTriangle)
{
// If so update camera view vector upon mouse movement (to make it fps-like)
m_Camera.MouseUpdate(glm::vec2(_event.motion.x, _event.motion.y));

// Check if mouse cursor is out of bounds, if so snap it back to the center of the screen
if (_event.motion.x > WIDTH - 10)
SDL_WarpMouseInWindow(m_MainWindow, WIDTH / 2, HEIGHT / 2);
if (_event.motion.x < 10)
SDL_WarpMouseInWindow(m_MainWindow, WIDTH / 2, HEIGHT / 2);
if (_event.motion.y < 5)
SDL_WarpMouseInWindow(m_MainWindow, WIDTH / 2, HEIGHT / 2);
if (_event.motion.y > HEIGHT - 10)
SDL_WarpMouseInWindow(m_MainWindow, WIDTH / 2, HEIGHT / 2);

}

// Normalised Device Coordinates
mouseX = (2.0f * _event.motion.x) / WIDTH - 1.0f;
mouseY = 1.0f - (2.0f * _event.motion.y) / HEIGHT;

// Check if user is picking triangle, if so set its position to mouseX and mouseY
if (bReplacingTriangle)
m_Transformation.SetPos(vec3(mouseX, mouseY, 0.0f)); // Note that m_Transformation is the transformation matrix for the triangle

break;
}
case SDL_MOUSEBUTTONDOWN:
{
// Get ray direction
glm::vec3 rayDirection = GetRayFromMouse();

// If the user has the triangle picked up already, then register its new location and turn off picking
if (bReplacingTriangle)
{
std::cout << "Placed Triangle at new X Coord: " << mouseX << ", Y Coord: " << mouseY << "
";
m_Transformation.SetPos(vec3(mouseX, mouseY, 0.0f));
bReplacingTriangle = false;
}

// Check for intersection with sphere and enable picking if collision occurs
if (ray_sphere(m_Camera.GetCameraPosition(), rayDirection, 0.5f))
{
bReplacingTriangle = true;
std::cout << "Picked Triangle at location X Coord: " << mouseX << ", Y Coord: " << mouseY << "
";
}
}

``````

Did you ensure that all your calculations are made with double precision ?

Yeah, still can’t pick it up from far away.

Because this is not double precision calculations. Plus, glm might use simple precision internally. Ensure this is not the case.

Also I just notice this:

``````float a = glm::dot(ray_direction_wor, ray_direction_wor);
``````

Are you use you have to use both the same vector for your dot product ?

glm already has a function that checks for line-sphere-intersection:
https://glm.g-truc.net/0.9.0/api/a00162.html#a33f0584acca58d7446daf594c3b8295f

try that first before implementing it yourself

“picking” objects can be done using a framebuffer with an additional integer texture attachment in which you draw the “int objectID”. “picking” is then just a read-back of the pixel on which the cursor is currently on.

it uses a “rgba color” value instead of an integer, and it does an additional render pass, you can avoid that easily by using MRT framebuffer with int texture attachment

you can avoid stalling the pipeline by reading the pixels value into a (double-buffered) pixel buffer object

Ha, I had no idea glm had intersection functions, that’s actually pretty cool :D. Though, I already got this far with my own ray-sphere intersection so I’m just gonna stick to it for now because it is working I just don’t know why these two test cases aren’t returning true upon intersection:

``````
if (x1 >= 0.0 && x2 >= 0.0)
return true;
if (x1 < 0.0 && x2 >= 0.0)
return true;

``````

I also liked the tutorial that you linked, but again I got pretty far with the trusty ray-casting so I’m going to stick to it for now. I’ll definitely give the other two a go for fun later on.

I think what I’m having is more of a calculation issue than anything else. I know this because when I debug my program and click on the triangle from far away, variable b squared - 4ac is greater than 0, which means that it has picked up an intersection, but then inside the if statement, it doesn’t return true for any of the real solutions (normally is a quadratic equation you get two real solutions when b squared - 4ac is > 0).

``````
bool PhysicsEngine::ray_sphere(vec3 ray_origin_wor, vec3 ray_direction_wor, float sphere_radius)
{
vec3 v = glm::vec3(m_Transformation.GetPos().x, m_Transformation.GetPos().y, m_Transformation.GetPos().z) - m_Camera.GetCameraPosition();
double a = glm::dot(ray_direction_wor, ray_direction_wor);
double b = 2.0 * glm::dot(v, ray_direction_wor);
double b_squared_minus_4ac = b * b + (-4.0) * a * c;

if (b_squared_minus_4ac > 0)
{
// Herein lies the problem
double x1 = (-b - sqrt(b_squared_minus_4ac)) / (2.0 * a);
double x2 = (-b + sqrt(b_squared_minus_4ac)) / (2.0 * a);

// Neither test true from far away
if (x1 >= 0.0 && x2 >= 0.0)
return true;
if (x1 < 0.0 && x2 >= 0.0)
return true;
}

return false;
}

``````

I literally got this from one of my old high school math book and translated into code.

Fixed the issue, thanks!

And in order to be as respectful as the posters who tried to help, may we know what was the issue and how you solved it ?

Sure thing, I didn’t think it’d be all that important since it’s just a math error and people tend to use libraries for collision detection, but here’s how I modified my function for whoever’s interested:

``````
// work out components of quadratic
vec3 v = glm::vec3(m_Transformation.GetPos().x, m_Transformation.GetPos().y, m_Transformation.GetPos().z) - m_Camera.GetCameraPosition();
long double a = glm::dot(RayDirWorld, RayDirWorld);
long double b = 2.0 * glm::dot(v, RayDirWorld);
long double b_squared_minus_4ac = b * b + (-4.0) * a * c;

if (b_squared_minus_4ac == 0)
{
// One real root
return true;
}
else if (b_squared_minus_4ac > 0)
{
// Two real roots
long double x1 = (-b - sqrt(b_squared_minus_4ac)) / (2.0 * a);
long double x2 = (-b + sqrt(b_squared_minus_4ac)) / (2.0 * a);

if (x1 >= 0.0 || x2 >= 0.0)
return true;
if (x1 < 0.0 || x2 >= 0.0)
return true;
}

// No real roots
return false;

``````

in case your are interested in a “simpler” solution:

a ray is defined by a fixed point A and a direction (unequal to 0) B
result = A + t * B

a sphere is defined by a fixed point C (center) and its (positive) radius R

step 1: find the nearest point on the ray to the sphere, called P:
take point A (of the ray) and point C (of the sphere), calculate the vector from A to C called “AC”

AC = C - A

then project that vector onto the ray:

P = A + |AC| * normalize(B) * cos(phi)

where phi is the angle between ray direction and AC, and |AC| the length of AC

make use of:
|AC| * normalize(B) * cos(phi) == AC * normalize(B)

==> P = A + AC * normalize(B)

you have A, AC, and B, so calculate P …

finally calculate the distance between P (nearest point on ray to sphere center) and C (sphere center)

D = C - P
distance = |D|

where D is the vector from P to C, and |D| = length of D

you have the distance of the closest point to the sphere on the ray, and the radius of the sphere, just compare:

if (|D| < R) that means intersection
if (|D| = R) that means the ray touches the sphere
if (|D| > R) that means the ray goes by without intersection

normaizing a vector means dividing it by its length
the length of a vector you get with pythagoras: length = sqrt(x^2 + y^2 + z^2)

summary:
ray: result = A + t * B
sphere: C, R

calculate:
AC = C - A
P = A + AC * normalize(B)
D = C - P

compare:
if (|D| < R) that means intersection
if (|D| = R) that means the ray touches the sphere
if (|D| > R) that means the ray goes by without intersection

thats more ore less what glm does …

Sorry for this question! I have trouble with picking to object by mouse.  