Ray picking in OpenGL ES 2


I was wondering if there was an easy way of taking the mouse X,Y coordinates and having the X,Z at Y=0 plane in game world (after projection and modelview matrix transformations).


Try to be a bit more specific/explicit about what you want.

You could find out where you are pointing if you impose some restrictions (fixed distance from the camera position) and/or build a ray from where the mouse points and use that later in your own collisions and/or calculate, based also on the stored z component the actual 3d pos that you’re pointing at, but this is generally not OGL specific, it’s either just generic math or more specialised, using the structures that you already (should) have in the game like camera and objects and stuff…

again, the question is too generic and, if I were to venture to provide an answer, it would be something like: sure, the math usually allows what you are trying to do (yeah, a generic answer :p)

There’s a gluLookAt camera, and a modelview matrix multiplied by a projection matrix (a generic setup :wink: )

The headache starts when I try to get the process backwards to get the gameworld x,y,z coordinates. The original thought was that if I have a constant Y plane, at Y=0, then maybe the process could be simplified… but there’s the projection matrix which, as far I understand, wont allow accurate values unless I counter it.

the site: http://eigenclass.blogspot.com/2008/10/opengl-es-picking-using-ray-boundingbox.html has an example I’ve tried to implement:

//Picking: (screen is 320x480)

unit_x = (touch.x - 160)/160;
unit_y = ((480 - touch.y) - 240)/240;

//	float(tan( FOV * PI / 360.0 ));
near_height = tan(60 * PI_OVER_360);

ray.x = unit_x*near_height*0.66666667;
ray.y = unit_y*near_height;
ray.z = 1.0f;

I understand that this part gives me “projection” values as to where the “touch” happened (something to counter the projection, but maybe I just got it all backwards…) . then goes the model view matrix inversion…

site: "…modelview matrix M, consisting of a 3x3 rotation submatrix R and a 3-element translation vector t " :

Matrix tmp;

tmp.m[0][0] = src->m[0][0];
tmp.m[0][1] = src->m[1][0];
tmp.m[0][2] = src->m[2][0];
tmp.m[0][3] = 0.0f;

tmp.m[1][0] = src->m[0][1];
tmp.m[1][1] = src->m[1][1];
tmp.m[1][2] = src->m[2][1];
tmp.m[1][3] = 0.0f;

tmp.m[2][0] = src->m[0][2];
tmp.m[2][1] = src->m[1][2];
tmp.m[2][2] = src->m[2][2];
tmp.m[2][3] = 0.0f;

tmp.m[3][0] = -1.0f/src->m[0][3];
tmp.m[3][1] = -1.0f/src->m[1][3];
tmp.m[3][2] = -1.0f/src->m[2][3];
tmp.m[3][3] = 1.0f;

memcpy(result, &tmp, sizeof(Matrix));

This is where I probably messed up. So, after the matrix inversion the vector multiplication happened. There’s the ray coordinates (x,y,z,0) which were calculated earlier and a start position {0,0,0,1}. Instead of having lots of unesseccery multiplications I did this part:

ray.x = invertedMatrix.m[0][0]*ray.x + invertedMatrix.m[1][0]*ray.y + invertedMatrix.m[2][0]*ray.z;
ray.y = invertedMatrix.m[0][1]*ray.x + invertedMatrix.m[1][1]*ray.y + invertedMatrix.m[2][1]*ray.z;
ray.z = invertedMatrix.m[0][2]*ray.x + invertedMatrix.m[1][2]*ray.y + invertedMatrix.m[2][2]*ray.z;

start_ray.x = invertedMatrix.m[3][0];
start_ray.y = invertedMatrix.m[3][1];
start_ray.z = invertedMatrix.m[3][2];

Now, I think this wasnt implemented properly. At all.
The values I get here of the ray coords itself are approximately between -1.2 and 1.2 at every value.

Now, when there’s need to iterate the vector (start position and direction) to the y=0 plane - how do I do it? Sadly, I’m not an expert at math.

I solved this eventually by porting the gluunproject from mesa,
modified it, and passed it twice for z=0 and z=1.
Then I compared to plane normal ( 0,1,0 ) at point 0,0,0 and got back the X and Z values.

I know my previously inverted matrix is wrong as hell