# Getting "real world" coordinates from screen

I’m trying to retrieve “the real world” coordinates when mouse is clicked in my 3d scene with gluUnProject and they are wrong…

1.I wonder if that my depth is 1500 and not 1 could be a problem?
(if yes how to fix it?)

2.My scene is rendered - “eye” and “look to” coordinates are changing all the time,so the same pixel can be 2 diffrent coordinates:
a)when z(depth) is greater and y(height) is 0
b)when z(depth) i smaller and y(height) > 0
Can gluUnProject deal with this properly?

3.My need is like this: I want to display in status bar the real world coordinates when user clicks mouse on the specific point on the screen ( but this action is not required , I mean application is not waiting for input, it is like an info feature)
Is gluUnProject standard solution for it?
What about using select or feedback mode (glRenderMode)?

Thanks

With “real world coordinates” you means the coordinate in your virtual environment?
You should know that your vertex are transformed by model-view and projection matrices into clip space and then divided by W to get the normalized device coordinate (NDC), this is very cool, cause till this stage openGL don’t care about resolution and it’s totally device agnostic.
So, you have your tip of the mouse that cover a pixel, this means that in the 3d space cover a whole line. If your windows size is Rx, Ry and your mouse coordinate is Mx My, in NDC your mouse is covering from the point
clipX = (Mx/Rx)*2.0-1.0;
clipY = 1.0-(My/Ry)*2.0; // the Y is usually upside down
clipZ = every Z from -1.0 to 1.0
You don’t know the Z
You have two possibility:
Solution 1
You can read the Z from the zBuffer
At this point you have all the clip coordinate and you can use gluUnProject (or make the computation by hand, is not that difficult)
Solution 2
call the gluUnproject two time, one with clipZ = -1 and another one with clipZ = 1, you get two point (in world space).
With these two point you get a ray and you can use some raycast algorithm to compute the mouse coordinate.

The first solution is easier but can be slow cause you use a get function.
With the second solution you need to check the ray against your geometry, if you have a good raycaster is not a problem.

So, yours questions:

1. Non sense, you are working in clip space.
2. I don’t understand, gluUnProject only make some matrix inversion, nothing more.
3. This is not an openGL question, witch kind of framework are you using? If you have the actual projection matrix, model view matrix and viewport you can call gluUnProject whenever you want.

glRenderMode is deprecated, maths will be never deprecated.

As Rosario said, your depth is screwed up. How are you getting it? gluUnproject is the correct command for you.

Normally this is setup by first calling glReadPixels on a 1x1 pixel on the depth buffer where the mouse clicks. If this isn’t between 0 and 1, something else is wrong.

You then use gluUnproject with the matrices used on projection, the viewport, and x,y, and z (mouse and depth) to get the “real world coordinate” back.

You should be able to do it in just two commands assuming you already saved off the required matrices…

gluUnproject(…)

My source code :
case WM_LBUTTONUP
{
dMousex = LOWORD(lParam);
dMousey = HIWORD(lParam);

``````    glGetDoublev(GL_MODELVIEW_MATRIX,modelMatrix);
glGetDoublev(GL_PROJECTION_MATRIX,projMatrix);

glGetIntegerv(GL_VIEWPORT,viewport);

dWinx = dMousex;
dWiny = (float)viewport[3] - (float)dMousey;
glReadPixels( dWinx, int(dWiny), 1, 1, GL_DEPTH_COMPONENT, GL_DOUBLE, &dWinz );

gluUnProject(dWinx,dWiny,dWinz,modelMatrix,projMatrix,viewport,&dRealx,&dRealy,&dRealz);
sprintf(szBuf, "X:%.2f Y:%.2f H:%.2f",dRealx,dRealz,dRealy );
SendMessage(hwStatus, SB_SETTEXT, 0, (LPARAM)(LPSTR)szBuf);

}
``````

1.What do you mean by “assuming you already saved off the required matrices…” ? Can’t I just get them every time I need with glGetDoublev(…) like I did?

2.Question regards Solution 2:
Why do I need to call the gluUnproject 2x, (clipZ = -1 ,clipZ = 1)?
What is returned z value when clipZ = -1 ?

3.I think I suspect that problem is
glReadPixels( dWinx, int(dWiny), 1, 1, GL_DEPTH_COMPONENT, GL_DOUBLE, &dWinz );
seems like dWinz always has the same value.

Thanks again.

1. Cause drivers architecture the glGet is a slow operation somewhere you set the project matrix and the camera matrix and the viewport so you should already know the these information. If performace is not an issue you can still use the glGet functions.

2. with clipZ = -1 you get the point on the near plane
with clipZ = 1 you get the point on the far plane. From these two points you can create a ray or a segment.

3. I’m not sure that you can use GL_DOUBLE in the read pixels, try with a float.

1. You can use glGetDouble. I meant that assuming you already did that, you only need to call the two functions I gave you.

2. I think the near plane should be 0. I gluUnProject uses window coordinates, which I think go between 0 and 1. Not sure if that matters…

3. As Rosario mentioned, I don’t think GL_DOUBLE is supported by glReadPixels. Even if it works, you might want to change it to GL_FLOAT just to be sure since that’s probably what you’re using if it did work anyway.

Thank you so much Stratto and Rosario!!
The problem was because of GL_DOUBLE in glReadPixels…It made the coordinates totally wrong.
Now they are correct,but a bit not accurate. Mistakes are like this:

for “world” x = 15 -> I got x = 15.2 (X are always most accurate ones)
for “world” y = 2 -> I got y = 1.83
for “world” z = -30 -> I got z = -28.8

Is there anything I can do to improve the accuracy?
My source code is like posted in my previous message (except the GL_DOUBLE fix )

BTW I didn’t call gluUnproject 2x with clips (-1,1).Seems like it’s unnecessary.What do you think?

One call for every position you read back. If you’re building a ray into your scene, you need two points, but if you just want one point, you just need one call.

How are you getting your world positions for these tests? Could you explain that?

The only thing I can think of is that your mouse coordinates don’t line up with the pixel sampling so you might need to offset them by a small amount.

I’m checking this now

For test application I just draw a small cube (radius = 1) wth coordinates x=15,y=2,z=-30, and then clicked a mouse on it’s center and checked if I’m getting these coordinates back… So it looks like your suggestion about mouse pixel sampling is the reason.

Thanks again