I’m looking for a formula or example that will choose values for gl_TessLevelOuter and gl_TessLevelInner based on the screen size of a triangle or quad. I do not want to make the user fiddle around with camera distance factors and other stuff like that. Do you know where I can find a good formula that is easy to integrate?
Your tessellation levels would depend on an estimate of the size of the actual surface your tessellation system is approximating. Without knowing what that surface is or what tessellation algorithm you’re using, there can’t really be an answer. Computing the appropriate levels for a bezier patch would require different math relative to something else.
I figured out a good formula that keeps the triangles at a certain size. I have TESS_SCALE set to 12.0:
if (gl_InvocationID == 0)
{
gl_TessLevelOuter[0] = 0.0;
gl_TessLevelOuter[1] = 0.0;
gl_TessLevelOuter[2] = 0.0;
gl_TessLevelInner[0] = 0.0;
//Calculate screen coordinates
screenvertex[0] = bufferSize.xy * (1.0 + gl_in[0].gl_Position.xy / gl_in[0].gl_Position.w) / 2.0;
screenvertex[1] = bufferSize.xy * (1.0 + gl_in[1].gl_Position.xy / gl_in[1].gl_Position.w) / 2.0;
screenvertex[2] = bufferSize.xy * (1.0 + gl_in[2].gl_Position.xy / gl_in[2].gl_Position.w) / 2.0;
//Discard offscreen triangle
if (screenvertex[0].x < 0.0 && screenvertex[1].x < 0.0 && screenvertex[2].x < 0.0) return;
if (screenvertex[0].y < 0.0 && screenvertex[1].y < 0.0 && screenvertex[2].y < 0.0) return;
//if (gl_in[0].gl_Position.z < 0.0 && gl_in[0].gl_Position.z < 0.0 && gl_in[0].gl_Position.z < 0.0) return;
if (screenvertex[0].x > bufferSize.x && screenvertex[1].x > bufferSize.x && screenvertex[2].x > bufferSize.x) return;
if (screenvertex[0].y > bufferSize.y && screenvertex[1].y > bufferSize.y && screenvertex[2].y > bufferSize.y) return;
//if (gl_in[0].gl_Position.z > 1.0 && gl_in[0].gl_Position.z > 1.0 && gl_in[0].gl_Position.z > 1.0) return;
//Get the edge lengths (this is using a triangle)
edgeLength[0] = length(screenvertex[1] - screenvertex[0]);
edgeLength[1] = length(screenvertex[2] - screenvertex[1]);
edgeLength[2] = length(screenvertex[0] - screenvertex[2]);
//Use edge length to calculate tessellation level
gl_TessLevelOuter[0] = max(1.0f, edgeLength[0] / TESS_SCALE);
gl_TessLevelOuter[1] = max(1.0f, edgeLength[1] / TESS_SCALE);
gl_TessLevelOuter[2] = max(1.0f, edgeLength[2] / TESS_SCALE);
gl_TessLevelInner[0] = max(gl_TessLevelOuter[0], max(gl_TessLevelOuter[1], gl_TessLevelOuter[2]));
}This algorithm has some flaws, centering around the fact that it doesn’t actually take into account your tessellation algorithm. That is, the method you use to compute the positions of the new vertices relative to the old.
For example, it is entirely possible for a tessellation where the control points are all off-screen to generate vertices which are on screen. If you’re using some PN-triangle technique for tessellation, you could imagine a triangle just to the edge of the screen with normals that lift the newly-generated vertices out of the plane of the triangle and on-screen.
In fact… it’s possible for an untessellated triangle to be on-screen when its vertices are off-screen:
% %
*------------
|
|
% |
|
All 3 vertices (the % character) are outside of that box, but as you can see, the diagonal of the triangle passes inside of the box. Oops.
Yes, I expanded the code to consider another three points that are represented by the vertex with max displacement applied. I suppose it could be possible still to have some false positives, but I’m satisfied. Note the code checks if all verts are outside one edge, not just if any are inside the viewport.
Of course if you are applying displacement then you will get cracks if neighboring polygons have differing levels of tessellation.
So I am thinking for model geometry I will use a distance-based formula with user-defined values for detail and near and far ranges. For terrain I will use something similar to the code above, but with calculation of neighboring tessellation levels to eliminate cracks.