Depth sorting with transparant billboards

[BwB]

This is just a wild shot in the dark here, but after reading all that hectic commotion I think what Theo might be asking is: How does OpenGL apply the modelview matrix to each vertex? If this is the case (I beleive me, I dont know why he would want to know that) then the answer is OpenGL rotates then translates. This can be proven by looking at the code for a vector * matrix (or matrix * vector depending on how you look at it) multiplication:

xOut = xIn * m[0] + yIn * m[1] + zIn * m[2] + m[3];
yOut = xIn * m[4] + yIn * m[5] + zIn * m[6] + m[7];
zOut = xIn * m[8] + yIn * m[9] + zIn * m[10] + m[11];

Where the translation in m[3] m[7] and m[11] gets added after the rotation has been performed.

Now, that being said, I could totally be wrong on my view of the question, and also look at my math above, it also looks wrong, but even if the matrix indices are wrong, the translation still gets added after the rotation. (And PLEASE correct me if I am wrong). This is the way I’ve been using it, and it seems to have been working

Now how this relates to sorting billboards I have no idea

[DFrey]

Not quite right. OpenGL doesn’t do rotations or translations in any particular order. It simply postmultiplies the current matrix by the rotation matrix (if you call glRotate), or the translation matrix (if you call glTranslate), or the scaling matrix (if you call glScale). You can easily translate before or after rotating. However, if you multiply the current matrix by a single arbitrary matrix, then yes, the “rotational parts” come first simply because of the order of operations. OpenGL has no control over the order of operations. But you do.

[imported_Theo]

Solved!

Someone on comp.api.graphics.opengl gave me the right answer. Its really obvious in hindsight and will make a lot of this discussion look stupid.

The plane equation! More specificly, the near plane.

Just find the distance of each object from the near plane and sort it. Works flawlessly now.

" //Begin Chart

[Math]          [OpenGL]           [OGLMeaning]

| 0 1 2 3 | | 0 4 8 12 | | vxX vxY vxZ px |
| | | | | |
| 4 5 6 7 | | 1 5 9 13 | | vyX vyY vyZ py |
| | | | | |
| 8 9 10 11 | | 2 6 10 14 | | vzX vzY vzZ pz |
| | | | | |
| 12 13 14 15 | | 3 7 11 15 | | sx sy sz sw |

OpenGL is the transpose of the Math ordering (rotated 90 degrees)

Labels:
vt where t can be x, y or z axis is the vector for t. The matrix stores vectors for each axis
pt is the x, y, z position
st is the x, y, z scale factor (magnify/shrink)
sw I forgot what it means

" //End Chart

This way people can work out many things for themselves.

You use the plane equation:

Ax + By + Cz + D = DistanceFromPlane

DistanceFromPlane = 0 only on the plane of course

To collect the depth sort data you do:

dim plA, plB, plC, plD, plX, plY, plZ as float
glgetfloatv GL_MODELVIEW_MATRIX, mf // mf = ModelFloats
plX = mf(12) // UnArrayified for code clarity, its not much faster
plY = mf(13) // although it is a tiny bit faster.
plZ = mf(14)
plA = -mf(2)
plB = -mf(6)
plC = -mf(10)
plD = -plAplX - plBplX - plC*plZ

A, B, C and D are the constants in the plane equation
X, Y, Z is a point on this plane. And also the camera origin.

So you collect the distance from each plane like

ObjDistFromPlane = objxplA + objyplB + objz*plC + plD

then you can sort these values and draw your objects in
order. It works flawlessly for me, and only one OpenGL
command is necessary is done per frame (glgetfloatv).

[msteinberg]

OpenGL matrices are no “transposes” of any normal matrix. They are simply differently organized in memory.

Second, the distance (the real distance) is
(p-r)*n

Your attempt should work, where ours should work too (we calculate the sphereical distance, where you calculate the planar distance, but for the visuals, that shouldn’t do any difference).

To fresh up your matrix knowledge:

Translation matrix:

1 0 0 a
0 1 0 b
0 0 1 c
0 0 0 1

Scaling matrix

a 0 0 0
0 b 0 0
0 0 c 0
0 0 0 1

Rotation matrices are too complicated here.