popping artifacts with shadow volumes and smooth shading

I just drew up a picture explaining what I was trying to do with words. It seems though that my computer is not seeing any usb devices right now, including my camera, which I took a shot of it with. Sorry. As for screenshots I cant get it working right now. Seems when I uncomment it the shadows disappear. sorry again. I had it a week or so ago when we were originally talking about this because you had me worried my shading code was wrong. I took out everything but the selfshadowing term. I did a csg union of a torus through a cylinder in 3ds (less than a 90 degree angle at where they join). I tested this and even this I saw errors. No matter how much theory we talk about matters much when you take a look at how it works in practice. If you’re saying you’ll just tessellate it so the error goes in to the selfshadow enough you’re doing to same as what you’d do to get rid of the original problem. In some test cases you wont see any problems. In the ones Ive tested I see them. maybe you could make a tool to correct models that might have errors. I dunno. What I can tell you is you cant just throw any model that works with per pixel shading at it and have them work. Thanks for the offer Cass.

Brian

Originally posted by cass:
[b]

I’m pretty sure you can identify these cases too (at some extra cost), and keep them from being lit, but let me think about it a bit more.

[/b]

yeah that would be cool. Are you talking while you shadow or preproccessing?

Edit:
Ok I just thought about that and if you could pull that off it would get rid of all but one case which is the s curve case. the one where the outer most triangle has the problem, ie the one on the silhouette.

[This message has been edited by zeroprey (edited 09-21-2002).]

can you explain that case to me? you’re not talking about the one in the picture PH posted - you’re talking about something else, yes?

as far as i know theres only one case. this is the one he mentions and the one ive been talking about all along. you have a triangle that is facing away from the light with a concave edge with normals facing the light. its in the shadow but at that edge its lit up because of the normals. its basicly the oppisite of the shadow popping.
_


/______|
.

i wonder if that works. if not try copying it to notepad. the dot is the light and the right long line is the face with the problem. the things sticking out are normals. the normal in the middle kinda faces the light (pretend it does) and the right face doesnt. that is the case i know of. there may be more. the reason why this is bad is that it is lit near the middle normal even tho this should be in shadow. is is a text atempt at the picture i drew but cant transfer. remember also that in 3d you have triangles that have small angles where the normal can be much further off from that of the face. Maybe this clears things up?

Brian

edit:
damn didnt work at all. anybody know how to do this here?

[This message has been edited by zeroprey (edited 09-21-2002).]

_


/______|
.

so the vert at the point of the V has the problem?

if so, is there a reason why the technique i suggested wouldn’t be robust?

in short - you use different normals for self-shadowing caluclations, and modulate the per-pixel lighting results to avoid the artifact.

[sorry - multiple edits]

to be more precise - u would send the triangles face normal at that V vertex, and use that to modulate the bump-map result. just means an extra 3 floats per-vertex probably. in most cases the self-shadowing normal would be the same as the vertex normal. pre-process identifies verts needing special treatment (V-case) and sends different verts for the tris on the two sides of the V - each with triangle face-normal as the “self-shadowing” normal. the same tangent-space basis is used for each vert, though, so non-shadowed lighting is OK.

[This message has been edited by vshader (edited 09-21-2002).]

vshader, this is the same problem PH posted. The problem is that even smooth surfaces are approximated by facets. Sure in this extreme case you can intuitively say you don’t want to average the normals, but this is just one example. The problem will arise with any situation where the inner shared normal is bent back towards the light. Now it may not be very bright in most cases however a crease will ALWAYS be created inside concave edges, even with subtle curvature because you go from stencil tested zero contribution to something however dim along the crease and it will switch on or off depending on the direction of motion. The trick is solving it considering the dynamic situation, sure you could use different normals and solve for a still, but at some point it’s going to switch between them.

Cass still has a nice idea, you just have to decide which artifact you like least. 1D texture modulation between 1 and 0 on polys where you flag the concave edge might fix this problem. But then you’d still have to flick that modulation on when going backfaced. You could start modulating before you become backfaced, this would be similar to the modulation term applied to fade the dot product to avoid back faced bump peturbations getting illuminated, you just pull it round the surface a bit, but the effectiveness depends on the tesselation of the model. Coarse models would require too agressive a fade.

[This message has been edited by dorbie (edited 09-22-2002).]

why can’t we just all raytrace, it would be so much simpler then

at least the major popping bug is now solved, thats a first step…

No, it wouldn’t work. If you raytrace it it will still fail the shadow test (stencil volumes produce the correct answer just an undesirable one). The flipped shadow is in a sense correct, but it’s the combination with the normal or color interpolation that’s wrong. The cause of the problem is with the faceted approximation of the curved surface. You’d have to raytrace a more detailed curved surface, and you could also sender such a surface conventionally.

OK - i get the picture. you don’t want to do a preprocess that involves looking at every possible position of a skinned mesh.

this would be similar to the modulation term applied to fade the dot product to avoid back faced bump peturbations getting illuminated,

exactly - you just send in an explicit normal instead of assuming tangent space <0,0,1>

still you could do it dynamically if you have a model structure that has adjacency info - which you would anyway for shadow volumes. you could identify backface/frontface pairs that might have the problem, and perhaps apply the technique that way. no chance of using tri-strips then, though, as anywhere a crease could form you need separate vertex indices for the triangles on each side of the edge so you can mess with the “self-shadowing” normals… or do you? if the frontface tri is shadowed anyway, does it matter which way it’s normal points?

depends wether u can quickly and reliably detect the problem cases at run-time.

damn… this idea is interesting, but b4 now i had infinite shadow volumes running totally on the GPU.

[This message has been edited by vshader (edited 09-22-2002).]

Maybe the following could work: use Cass’ approach on the triangles in the backfacing list ( wrt. the light ) that are on the silhouette ? These are really the only problematic triangles.

Edit:
Hmmm, no still wouldn’t work.

[This message has been edited by PH (edited 09-22-2002).]

Since we’re working on-the-fly here, let me try to define a term that may be helpful.

Let’s define the concavity of an edge as follows:

If for each triangle, ABC, we keep an index to the vertices A, B, and C, and we also keep an index to the vertex opposite A, B, and C as a, b, and c. Here’s some ascii art.

  C
 /|\
/ | \

/ |
A—B—a

That is, vertex a is the vertex on the adjacent triangle that is opposite edge BC.

We can say that edge BC is concave if dot(plane_ABC, a) is positive, and edge BC is convex if dot(plane_ABC, a) is negative.

Ideally we could just say that we also shadow polygons where the following conditions apply:
a) back facing the light
b) have a concave silhouette edge

The ambiguous case is when a polygon has two silhouette edges, where one is convex and one is concave. It’s tempting to call this poorly tessellated geometry, but for concave surfaces, one of the possible silhouette loops must make a transition from concave edges to convex edges. Seems reasonable that this could happen at a single polygon.

<processing…>

Cass

PS minor wording correction

[This message has been edited by cass (edited 09-22-2002).]

Another on-the-fly idea : In addition to the idea I posted above, only use the vertex normals with the silhouette vertices. For vertices ( on backfacing triangles on the silhouette ), use the triangles normal.

Sounds reasonable ?

Edit:
Or simply set the shading on those vertices to 0.

[This message has been edited by PH (edited 09-22-2002).]

Originally posted by dorbie:
No, it wouldn’t work. If you raytrace it it will still fail the shadow test (stencil volumes produce the correct answer just an undesirable one). The flipped shadow is in a sense correct, but it’s the combination with the normal or color interpolation that’s wrong. The cause of the problem is with the faceted approximation of the curved surface. You’d have to raytrace a more detailed curved surface, and you could also sender such a surface conventionally.

i don’t see any point why raytracing would fail…
you find out what you hit nearest on screen. once you have this point, you check if there is anything between that point and the lightsource. if that is true, you have a factor 0 for the shading part of this light, if you did not found anything, you have a factor 1. that one you can multiply with the normal shading then…

all you need is to get your intersections right…

problem here is, items get “shadowed twice”, once thanks to the shadowvolumes, once due the shading. that does only affect selfshadowing, and generates that popping. once you tweak yourself around this, you get other artefacts, seen in here…

about the selfshadowing… you don’t test if you hit the triangle with the to_light-ray for finding out if there is something between the point and the light… problem solved perfectly…

This illustrates my idea,
http://www.geocities.com/SiliconValley/Pines/8553/noPop_Problem.html

The dark edge is from self-shadowing. Green lines are normals, yellow dot is the light.

Not drawn to scale .

Paul,

Seems reasonable to me for handling the case where a back facing triangle participates in both a convex and concave silhouette edge.

My preference would be to handle it at the polygon level only rather than both the polygon and vertex level if possible.

                      Thanks -

Cass

Dave, the result would be indistinguishable from the stencil approach. The WAR would suffer from the same problem. Perhaps ybe I’m missing something you are proposing.

Originally posted by cass:
Ideally we could just say that we also shadow polygons where the following conditions apply:
a) back facing the light
b) have a concave silhouette edge

but we don’t want to shadow the whole tri - just the offending edge. the other vertex should have a nice smooth shaded gradient.
we just want to id the offending edges and send some info that will “unshade” them…
and if i get the problem right (i can’t implement anything 'cos my engine is broken) - it’s not the silhouette edge thats concave, but a non-silhouette edge of a triangle on the silhouette edge - only one vertex of an offending triangle need be on the actual silhouette edge, and the opposite edge will possible exhibit the problem depending on normals.

[b] originally posted by PH
Another on-the-fly idea : In addition to the idea I posted above, only use the vertex normals with the silhouette vertices. For vertices ( on backfacing triangles on the silhouette ), use the triangles normal.
Sounds reasonable ?

[/b]

thats pretty much what i’ve been suggesting, but you describe it much more cleanly than me. we send a normal that correctly describes the “backfacing-ness” of this tri for use in the shading calculations - a “self-shadowing” normal. there’s easily room to fit this normal in a color interpolator and use it instead of lightspace Z for the self-shadowing modulation of per-pixel lighting results (assuming tangent space bumpmapping)… i get the feeling the object-space case would be easier, but i haven’t thought it thru properly

edit: oops UBB code

[This message has been edited by vshader (edited 09-22-2002).]

vshader and Paul,

I agree that the “right” approach is to shade-as-black only the vertex that is on the concave silhouette, and only for the polygons that are back facing the light.

The only time you really need to draw those polygons at all is when there is an inflection point (concavity change) inside the polygon. That is, when the edges are not all convex or all concave.

In any case, I think you can identify the “bad” case and fix it. The question now is, how to do it efficiently.

Cass

Originally posted by dorbie:
Dave, the result would be indistinguishable from the stencil approach. The WAR would suffer from the same problem. Perhaps ybe I’m missing something you are proposing.

yes, you are. and i’m missing what…, so i can’t help you

i know just one thing, never had any such problems with raytracing at all… but i had them the first time i tried to work with stencilshadows, so no, they don’t give the exactly same results…