I am trying to grasp the math behind the clipping projections for glFrustum and glOrtho. I have their matrices from the MSDN website and am trying to go over by hand examples to see how a vector changes. However I am not coming up with what I thought would be the correct final vector and so I’m getting really confused with what I’m not thinking about correctly. I’m going to go through my thought process. Tell me where I’m going wrong. So taking just glOrtho for simplicity there are two transformations that occur, a Translation and a Scale, which when multiplied give us S*T which is what MSDN has. Now I’ve tried to, with an unchanged modelview(so it’s an identity matrix) to keep it extremely simple, multiply the projection matrix by some vector to see its output vector which should have x y z components in the range of -1 to 1(provided the vector is within the viewing volume). I believe these are called normal device coordinates, or they well be after normalizing w out. So lets say our ortho volume has values of t = 1, b = -1, l = -1, r = 1, n = 2, f = 4.
Now from what I understand the projection matrix will take this volume and translate it around the origin and scale it so it’s -1 to 1 on each axis. this is already scaled correctly as its in the form of a 2x2x2. So I would think that vector (0,0,-1,1), which should be the center of the face on the near plane, would get transformed and become (0,0,1,0). Is this assumption correct? Because when I do the math it doesn’t come out right and I don’t know why. The z component becomes z’ = -2/(f-n)*z - f+n/(f-n)*w = -(-2) - 6/2 = 2 - 3 = -1. Now I was expecting that to become 1 and not negative 1. Why am I getting this for a calculation? Does the final projection reflect the original? I tried it with the farther z value -4 and ended up with 1 which is what I wanted for the first vector. This doesn’t appear to be a transform and scale though, as if I only translated it I think I’d end up with the same answers as there’s no actual scaling to be done. Could someone clarify this? I’m having trouble with Frustum as well, but I feel if the glOrtho example can be explained then I can figure out what I’m doing wrong with glFrustum.
I’ve figured out that it’s just the translation matrix that I can’t seem to follow correctly. I’ve scaled vectors whose viewing volume was centered around the origin with the scaling matrix (2/(r-l), 0, 0, 0, 0, 2/(t-b), 0, 0, 0, 0, 2/(f-n), 0, 0, 0, 0, 1). So it seems the issue is with the translation matrix.
Edit: I’ve figured out my issue. Found out that after the projection matrix get multiplied the new clipping/NDC matrix is left handed and not right handed, so the calculations are correct. My professor of course had a right handed clipping plane being shown so that caused confusion.