I’m looking to find where on a square the mouse is pointing.
The projection matrix is gluPerspective(60,ratio,0.1,1000);
The modelview matrix is gluLookAt(0, 10, 10, 0, 0, 0, 0, 1, 0);
The viewport is (0, 0, 800, 600).
The corners of the square are (-5, 0, -5) and (5, 0, 5).

I tried using gluUnProject but I realized that I don’t know the window z coordinate. So I tried making a vector by calling gluUnProject twice using window coords of 0 and 1 then interpolating for y=0 but that didn’t seem to work. Should it have? Is there a better way? Any help is appreciated.

You can use gluUnProject. The window Z-coordinate is the value you read from the depthbuffer. Use GL_DEPTH_COMPONENT (I think) with glReadPixels to extract the depthvalue for gluUnProject.

The objy is usually very close to 0. I changed my near and far viewing planes to 4 and 32 because I thought it might be losing precision in the depth buffer but it’s still off by some. I’m pretty sure I’m sending the right matrices. Is there normally error or am I doing this wrong?