I know this must be described somewhere, but I can’t find it…
I need to be able to get the screen-space coordinates of a submitted vertex (in the Vertex or Fragment shaders) so I can generate a screen-space UV coordinate from it. I should be able to transform the vertex by multiplying it by gl_ModelViewProjectionMatrix * gl_Vertex or ftransform() correct?
Any attempt to project the vertex, gives me results that are close to the desired behaviour, but not stable. Is there a link anyone knows of that describes exactly what these transforms are supposed to produce?
Essentially I am trying to implement an “indirect texture lookup” solution with my deferred shader engine, to allow warping of previously rendered pixels.
I’m not sure what you’re trying to do exactly, but here’s some info that may help:
Both Cg and GLSL provide screen space coordinates in the fragment shader.
The variable gl_FragCoord is available as a read-only variable from within fragment shaders and it holds
the window relative coordinates x, y, z, and 1/w values for the fragment
If you want to manually calculate the coordinates in the vertex shader, be sure to divide the coordinate with the q-value (homogeneous coordinate), otherwise it won’t be linear in screen space.
Transforming a vertex by gl_ModelViewProjectionMatrix * gl_Vertex transforms it into clip coordinates. By dividing x, y and z of the clip coordinate by its w, you get normalized device coordinate. In this coordinate system x, y and z range from -1 into 1. That is, (-1, -1, -1) is the bottom-left coordinate at the back plane of the clip volume. Without this perspective division, as -NiCo- stated, coordinates won’t be in the normalized [-1, 1] range.
Normalized device coordinate may be the thing you can use in your shader, but you can also transform this into viewport coordinates (also called window coordinates) if needed, which is in the range [0, width] for x and [0, height] for y, and for z it is whatever you specify in glDepthRange.
I would also recommend you to read section 2.11 Coordinate Transformations of the OpenGL specification. Very interesting stuff!