gluUnProject() is giving me trouble

OpenGL OpenGL: Basic Coding
1

I’m using gluUnProject() to try to determine the world coordinates of the cursor position. I’m using a perspective projection set up with gluPerspective() with the near clipping plane at 2 and the far clipping plane at 100. I’m attempting to find out where the cursor is assuming z = -40. I tried passing a winz value of (40-2)/98 = .3878 to gluUnProject, but it doesn’t work. The x and y values it returns are much too small. Anyone have any idea how to get the right values?

EDIT: Here’s the function in question:

inline void convertCoords(double x, double y, double z, double *nx, double *ny, double *nz)
{
	double modelview[16], projection[16];
	int viewport[4];

	glGetDoublev(GL_MODELVIEW_MATRIX, modelview);
	glGetDoublev(GL_PROJECTION_MATRIX, projection);
	glGetIntegerv(GL_VIEWPORT, viewport);

	double winz = (z-2.0)/98.0;
	double winy = viewport[3] - y - 1;

	gluUnProject(x, winy, winz, modelview, projection, viewport, nx, ny, nz);
}
2

The mapping from world Z to depth buffer Z is non-linear, so a linear interpolation will not give you the correct winz value. For the given setup, a world Z of -40 corresponds to a winz value of about 0.9694.

Here’s a formula for converting world Z to depth buffer Z. It’s from a MATLAB-script I have.

%in:  wZ       - world Z
%in:  near/far - near/far clip plane
%out: dbZ      - value written to depth buffer

a = -(far - near) / (2 * far * near);
b =  (far + near) / (2 * far * near);

dbZ = -1 * (wZ*b - 1) ./ (wZ * a);
dbZ = 0.5 * dbZ + 0.5;

Just one important note. wZ should be positive here, so for a depth of -40, set wZ to 40. Or just change the formula to take negative values instead if you prefer that :slight_smile:

edit: variable name changed, see post below

3

I’m actually having this same problem. I haven’t thought of a way to fix it yet (no time to work on it, sigh), however, I believe it has to do with UnProject returning the viewport coordinates, rather than opengl coordinates (like, I’m trying to work on the modelview coordinates).

Hopefully someone more experienced will come up with an answer, but if I figure it out, I’ll be sure to post it here.

4

Bob, I tried the formula and it didn’t work. Perhaps I don’t understand MATLAB syntax very well, or maybe I typed it in wrong. Could you explain what ./ means, and what fbZ is? Are those typos, or some MATLAB thing I’ve never heard of?

EDIT: .9694 doesn’t seem to work either. .9539 seems to work well…

5

fbZ was a misprint, and should be dbZ. This is not the exact original code (although it works, I have tried). I changed the variable name first to fbZ (as in frame buffer Z), but then changed my mind and changed it to dbZ (as in depth buffer Z), and it seems like I forgot to change it everywhere. Stupid to change it for no reason like this, I know. So fbZ should be dbZ.

./ is correct though, and it means element by element division and is used to vectorize code. Basically, it’s normal division when working with scalars, and that is what you should use. When vectorizing code like I did, you don’t have to evaluate the function several times, one for each value of wZ; you just make wZ an array of all values you want evaluated, and MATLAB automatically performs the calculation element by element.