Depth Component of Converting from Window -> World Coordinates

I’m working on a program that draws a 100x100 grid and allows the user to click on a cell and change the color.

Currently I have a Gird & Cell C++ class that I can initialize in whatever dimensions I want. Cells are stored in a 2D vertex. Each display() function call, the cells are looped through and drawn to the screen.

Clicking also works currently, however only when looking at the grid face on (i.e. viewDirection.y value of -90.0).

What I’ve been stuck on the last few days is selecting the correct cell when looking at the grid from an angle.

Any help would be appreciated. Code snippets below:

Vertex Shader:

#version 330

layout(location = 0) in vec4 position;

smooth out vec4 theColor;

uniform vec4 color;
uniform vec3 offset;
uniform mat4 perspectiveMatrix;
uniform mat4 viewMatrix;

void main() {
  vec4 cameraPos = position + vec4(offset.x, offset.y, offset.z, 0.0);

  gl_Position = perspectiveMatrix * viewMatrix * cameraPos;
  theColor = color;

(I know this needs to be improved- setting uniforms for each cell that is rendered probably is definitely not the most efficient way to do it!)

perspectiveMatrix calculation:

float aspect = 1.0f;
float far = 100.0f;
float angle = 90.0f;

glm::mat4 perspectiveMatrix()
  return glm::perspective(angle, aspect, 0.1f, far);

viewMatrix calculation:

glm::vec3 position(0.0f, 3.0f, 0.0f);
glm::vec3 viewDirection(0.0f, -1.0f, -1.0f);
const glm::vec3 up(0.0f, 1.0f, 0.0f);

glm::mat4 WorldToViewMatrix()
  return glm::lookAt(position, position + viewDirection, up);

Click function:

void mouse(int btn, int state, int x, int y)
  if (state == GLUT_DOWN) {
    GLint viewport[4];
    GLfloat winZ;

    glGetIntegerv(GL_VIEWPORT, viewport);
    glReadPixels(x, y, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ);

    glm::vec3 screen = glm::vec3(x,viewport[3] - y, winZ);
    glm::mat4 view = WorldToViewMatrix();
    glm::mat4 projection = perspectiveMatrix();

    glm::vec3 pos = glm::unProject(screen, view, projection, glm::vec4 (0, 0, viewport[2], viewport[3]));

    if (btn == GLUT_RIGHT_BUTTON) {
      grid.cells[(int) pos.x][(int) (pos.z -1.0f) * -1.0f].changeColor(1);

(just don’t click outside of the grid!)

You could give a try to ray/triangle intersection algorithm, for example Moller-Trombore.
First you have to calculate ray direction, then using algorithm mentioned above you can get the distance of the point you’ve clicked at. Multiply the ray times the distance and you got exact position.