I have just implemented Z-Fail stencil shadow volumes, but I have run into a few snags.
The first is that when I move the ‘camera’ between the light source and the shadow, the shadow either disappears or has a hole form in it, letting through the original unshadowed colouring. For example, I have one sphere casting a shadow onto a wall. Upon moving between the sphere and the light source a hole in the shape of a sphere will appear in the shadow on the wall.
The second is that alot of model shapes I load in, will cause abnomalities in my shadows. The shadows will have gaps, rotate about and stretch. I have been reading around and thought this may have to do with non-closed models. But it happens on closed objects. Spheres and cylinders appear to work fine, but cubes and other cubish shapes cause this problem.
I am doing my vertex extrusion in a GLSL vertex shader and it appears to be working. But I cannot find what is causing these two problems. Any help as to what might cause these would be appreciated.
The first is that when I move the ‘camera’ between the light source and the shadow, the shadow either disappears or has a hole form in it, letting through the original unshadowed colouring…
This should not happen when using the z-fail technique in the right way. The main reason for using z-fail is that shadow volumes get clipped by the front clipping plane and the shadow calculation (adding/subtracting using stencil) using z-pass leads to incorrect results.
Do you draw the front and back cap of the models (using a small polygon offset to prevent z-fighting)? This is important for z-fail to work.
Do you extrude the silhouette edges to infinity as proposed by Lengyel (http://www.gamasutra.com/
features/20021011/lengyel_pfv.htm)? If you do it this way, be sure to use a perspective projection matrix with an infinite far plane or use GL_DEPTH_CLAMP to avoid the back cap being culled by the far clipping plane. Good explanations can be found in the above paper or in another good paper written by Everitt and Kilgard “Optimized stencil shadow volumes”, that is available on the nvidia website.
The second is that alot of model shapes I load in, will cause abnomalities in my shadows…
Are you sure your models are two-maifold? Meaning that every edge is shared by exactly two triangles? That means that there should be no T-vertices or whatsoever. One paper on this topic is the one from Aldridge, and Woods: “Robust, Geometry-Independent Shadow Volumes”
Originally posted by Swiftless:
[b]
But it happens on closed objects. Spheres and cylinders appear to work fine, but cubes and other cubish shapes cause this problem.
I am doing my vertex extrusion in a GLSL vertex shader and it appears to be working.[/b]
On which geometry do you apply the extrusion shader? This behavior looks to me as if you applying it on the geometry of the original model. Doing so will deform the outline of the geometry which is more visible on objects where each vertex is important for the shape (e.g. cube).
Various examples that do extrusion in vertex shader use modified geometry where each edge is replaced with two triangles. I think there was some old paper about that on Nvidia web.
GuentherKrass, I am extruding to infinity in a perspective projection matrix, but I am not using GL_DEPTH_CLAMP, I will add it in and get back to you. And yes, the models are two-manifold.
Komat, I will have a look for that paper as I am currently exruding the original model. I wouldn’t have expected the original model to cause these problems.
Originally posted by Swiftless: Komat, I will have a look for that paper as I am currently exruding the original model.
I was unable to find the paper I was talking about, maybe it was part of slides from some conference, however I found two descriptions of generation of the geometry. One with pseudocode at ATI site and one with images and textual description in DX documentation .
Make sure you have a render mode that doesn’t do the extrusion in the shader. It’s just easier to make sure you know exactly what’s happening if you do all the computations on the CPU as a fallback. That way you have a “truth” pass.
It sounds like your models aren’t closed, or they have inconsistent windings. If you render the original models with face culling on, do they look right? Are there any open edges?
Making sure that your model really is closed (every edge is shared by exactly two triangles), and that all the faces are wound consistently is a really important thing to be able to check for.
To make sure your algorithm is rendering correctly, you could start with a simple shadow caster like a tetrahedron. 4 vertices, 6 edges, 4 faces. Pretty easy to verify the model is correct.
Originally posted by cass: Making sure that your model really is closed (every edge is shared by exactly two triangles), and that all the faces are wound consistently is a really important thing to be able to check for.
Actually that is just a myth that they need to be closed, if the engine is designed correctly the models only need to be watertight(where polygons only need to overlap a little with each other, not connect) and not closed.
(side note: yes my engine can and often do shadow single unconnected polygons, though the shadows can look a bit off if viewed from the wrong viewpoint because the back of the polygons does not cast shadows)
Polygon winding order is more important though, one should make sure that the rear cap is wound correctly (ie. reversed from the front cap).
It would be nice if you posted a screen shot or two of the problem, my guess is that then we could tell you what’s wrong.
Ok, I managed to fix the first problem by enabling GL_DEPTH_CLAMP_NV (Does GL_DEPTH_CLAMP exist or does GL_DEPTH_CLAMP_NV work on ATI cards?)
As for the second problem, I will check all my windings, but so far I believe each face has been wound the same way. Also, the models are watertight. I will grab some screenshots as zeoverlord requested in a few mins.
Ok, I have some images and have tested with a few more shapes and models.
Cube viewed practically from light position.
Cube viewed from an angle.
Nvidia Knight model from Stencil Shadow Project.
I have also tried the glut cubes and teapot, but to no avail. If this is a problem with the model, can someone send me a test model?
Also, I am smoothing my normals for my model, could this lead to inconsistencies when calling gl_Normal in my vertex shader to check if it is facing the light, as it is not the surfaces true normal, it is a normal generated by the smoothing of the surrounding normals? The thought cropped up when playing with the normal in the vertex shader.
I had something like this once, in my case it had to do with bad data generated by my connectivity function that causes some polygons to cast silhouette polygons that are wound the wrong way and are in the wrong place.
Normals: you should always have four normals on hand, one face normal used for stencil shadows and three vertex normals for shading that may or may not be interpolated.
Ok, I managed to fix the first problem by enabling GL_DEPTH_CLAMP_NV (Does GL_DEPTH_CLAMP exist or does GL_DEPTH_CLAMP_NV work on ATI cards?)
On ATI cards you can think of some clever function to map z-values to the [0,1) range and modify the projection matrix accordingly as described in the Lengyel paper. Another idea is to clamp the z-values manually in the fragment shader, but I’m not sure if this works well in practice.
On to the second problem:
I don’t know exactly how your extrusion shader works, but I think the problem is here:
Also, I am smoothing my normals for my model, could this lead to inconsistencies when calling gl_Normal in my vertex shader to check if it is facing the light, as it is not the surfaces true normal, it is a normal generated by the smoothing of the surrounding normals?
So apparently you check each face whether it is a possible silhouette edge and extrude the vertices if the triangle shares a silhouette edge.
In which space are you doing this? You said you’re using an infinite projection matrix, so the best way would be to check the facingness in clip space by multiplying the normal with the inverse of the transposed of the projection matrix in the vertex shader. Then you only have to check if the z value of the transformed normal n’_{z} is positive (facing away) or negative (facing towards the camera), since the camera looks down the negative z-axis.
