Unprojecting mouse cursor to find relative position to a certain object (OpenTK C#)

Hi,

I have a player object that’s moving around the screen. The camera is a fixed camera, looking down on the player (like in Diablo).

Now I want the player object to rotate towards the mouse cursor. The player is not always on the center of the screen (for this, I already have a solution).
In order to do this, I think I need to project the mouse cursor to the same height (y-axis) that my player is on (y-axis is “up” in my game).

So far, I have this code:


private bool Unproject(float winX, float winY, float winZ, out Vector3 position)
{
        position = Vector3.Zero;
        Matrix4 transformMatrix = Matrix4.Invert(World.CurrentWindow.GetViewMatrix() * World.CurrentWindow.GetProjectionMatrix());
        
        Vector4 inVector = new Vector4(
            (winX - World.CurrentWindow.X) / World.CurrentWindow.Width * 2f - 1f, 
            (winY - World.CurrentWindow.Y) / World.CurrentWindow.Height * 2f - 1f, 
            2f * winZ - 1f, 
            1f
            );
           
        Matrix4 inMatrix = new Matrix4(inVector.X, 0, 0, 0, inVector.Y, 0, 0, 0, inVector.Z, 0, 0, 0, inVector.W, 0, 0, 0);
        Matrix4 resultMtx = transformMatrix * inMatrix;
        float[] resultVector = new float[] { resultMtx[0, 0], resultMtx[1, 0], resultMtx[2, 0], resultMtx[3, 0] };
        if (resultVector[3] == 0)
        {
            return false;
        }
        resultVector[3] = 1f / resultVector[3];
        position = new Vector3(resultVector[0] * resultVector[3], resultVector[1] * resultVector[3], resultVector[2] * resultVector[3]);
        
        return true;
}


Now I unproject the mouse cursor once for the near plane (winZ = 0) and the far plane (winZ = 1).


protected Vector3 GetMouseRay(MouseState s)
        {
            Vector3 mouseposNear = new Vector3();
            Vector3 mouseposFar = new Vector3();
            bool near = Unproject(s.X, s.Y, 0f, out mouseposNear);
            bool far  = Unproject(s.X, s.Y, 1f, out mouseposFar);
            Vector3 finalRay = mouseposFar - mouseposNear;
            return finalRay;
        }

My problem is:

How do I know if the values are correct. The values in the “finalRay” Vector are quite small - always. I would have thought that i would get much bigger z-values because my near plane (perspective projection) is 0.5f and my far plane is 1000f.

And how can I find out if the mouse cursor is left/right (-x, +x) or behind/in front of (-z, +z) the player? (I know the player’s position)

Where is my error?

Cheers and thanks in advance!