Curved Relief Mapping with correct silhouettes

Originally posted by SirKnight:
[b] [quote]Originally posted by knackered:
Brilliant.
How does it work with shadow volumes? :slight_smile:

“Shadow volume reconstruction from depth maps” maybe?

-SirKnight[/b][/QUOTE]I think rendering this shader from the point of view of multiple lights to get the depth maps for subsequent edge detection may perhaps start to negate the vertex throughput savings achieved.
Maybe in a couple of years…

That’s very cool. I have been following your work rather intently lately since it fits quite closely with my own research.

In developing a very similar technique for displacement mapping based on ray tracing I ran into the same problem: silhouettes are completely flat.

And I arrived at a similar solution to the one you have just mentioned (and was used for Wang et. al.'s View-dependent Displacement Mapping): generate principal curvatures per-vertex on the mesh, and use these to approximate a ray in texture space by a parabola instead of a straight line.

I found that the parabolic approximation worked very well on spheres (as you presented) and even on cylinders. But then I found that it failed on a torus. As far as I could tell this was not a bug in my code: it seems that an approximation to curvature based on local derivatives is actually quite poor, particularly in places such as saddle points. Have you had similar results with the torus? Could you perhaps post a picture? It would be great to see it on more complex models.

I believe that Wang et. al. realized the drawbacks of the parabolic ray approach, which is probably what led them to abandon it for their EG04 paper.

Edit: I had a version of the GDM paper without pictures, so I missed that figures 8c and 8d show clearly that VDM did in fact have the same problem with silhouettes that I had:
http://research.microsoft.com/users/xtong/xtong.html
I guess this also shows how easy it is to hide crappy results in a paper. :rolleyes:

Do shadows still work with this new addition?

-SirKnight

Hi Pragma, looks like we had similar ideas so. Do you also use a linear/binary search for the ray intersection?

And yes I used a parabola for the ray intersection as x*x is the simplest function to define the curved ray through texture space.

I made this demo the day before I had to fly out of Brazil to US for a few business meetings. So I did not have time to test it with other objects.

As soon as I get back home next week I will do some more work for general meshes and also the torus case. I thought in using negative curvatures for parts of the torus.

The bad thing on VDM and GDM is the 4D map that needs to be generated. This reduces the quality as sampling angles with low res textures gets you artifacts when looking too close. It is much simpler for artists to use standard depth maps.

Maybe the GDM approach could use a ray intersect like the one in my shader (linear/binrary serach) instead of sampling the 4D map.

And SirKnight, my new shader can also do self shadows and correct depth values even more precisely than previous version as depth range is better defined in it (previous one depth was an arbitrary factor and new one depth factor is in scene units).

I am actually not using linear/binary search. I started by implementing linear search, but you run into the issue of aliasing pretty fast. Essentially with linear search there is always a chance you miss a detail completely. I am using a technique based on John Hart’s “Sphere Tracing” ( http://graphics.cs.uiuc.edu/~jch/papers/ ) . It is sort of a compromise between 4d/5d vdm/gdm and just 2d depth maps, since you need to keep a 3d map.

The choice of ray intersection technique is somewhat orthogonal to the way you define your ray, so I am sure that the GDM approach would work with linear search. This is basically what was done by Hirche et al ( http://portal.acm.org/citation.cfm?id=1006077 ) as well as Peng et. al. ( http://portal.acm.org/citation.cfm?id=1015773 ).

Hi again Pragma… sorry for the long delay on getting back to this shader… was too busy at work recently.

But I found some time to generalize my curved relief mapping idea yesterday and was surprized by the excelent results I got!

I use a quadric to represent curvature (A.X^2+B.Y^2=Z) and only pass coefficients A and B at each vertex. To calculate the quadric coefficients for each mesh vertex I use a simple least square matrix solver just like shown on the paper garimella-2003-curvature.pdf (google for it).

I already tested my new shader with a sphere (constant positive curvature in two directions), cylinder (constant positive curvature in one direction and 0 on the other), torus (constant positive in one direction and positive/planar/negative in the other) and a teapot (all different curvatures all around).

I will post a demo/paper asap. Screenshots by the end of the day.

That’s pretty nice looking. There were some issues I noticed in that screenshot with the wood texture and the pyramids. That one pyramid on the left is quite “stair steppy.” Other than that, everything else looked pretty good. BTW, how many instructions and all that did the old version have compared to your new version?

-SirKnight

The images look great! :slight_smile:
When can we expect more imformation or a paper? :stuck_out_tongue:

Hi… just finished support for shadows with the relief mapped objects.

Shadows from relief object into world have the nice silhouettes and shadows from world to relief object project correctly over the per fragment displaced surface.

NormalMap shadows
Curved ReliefMap shadows

Also I made a new video showing the effect in action with camera/object movement. I also included the actual geometry being drawn in a red wirefarme so you can see exactaly what is going on.

Curved relief map video

Enjoy! Paper and demo soon…

It looks great, but i noticed an artifact:
when a stone goes into the distance it wobbles slightly (artifact seems to be stronger near the triangle edges)
you’d probably notice it more when using a more regular texture
the artifact is subtle however, and i doubt it would be noticable in practice…

looks very nice.

but whats the performance?
compared to parallax mapping and other techniques?

also compared to a displaced high geometry version of the object?

did you test it with more arbitrary meshes?

how about inwards looking meshes (eg cornell box), how well does it perform at the seams?

cheers
Volker

Yes there are some artifacts on sphere top/bottom as mapping there compresses too much. Also the top/bottom vertex degenerates the mapping completely. I’m working on having better support for such degenerated vertices in my tangent space computation routine. Also I’m testing ways to compensate for texture mapping expand/compress in shader.

The shader is much more complex than simple bump/parallax mapping. After all optimizations I could think about the shader compiles to 220 assembler instructions and uses 6 registers (0 to 5).

Hello,

dramatic progress, amazing pictures.

How do you handle sharp edges? E. g. a cube or actual level geometry (corner of a brick wall - the ray that is cast on the texture has to change direction).

Would it be possible to have a look at your code?

Regards,

mbue

Really impressiv ! Good job !

Gaby

any more progress on a demo?

my browser only shows one page for this thread, but “fpo’s” user profile says he recently posted #105 here… i’m seeing if posting will get me the other pages.

edit: is there something funny about this thread, or just the bbs database…

FPO’s user profile says:

4 new Relief Mapping shader (better Parallax Mapping?!?) (post #000105) OpenGL coding: advanced 03-19-2005

there are not 105 posts here or anything close… was a big chunk of this thread deleted?

anyhow, i’m just trying to bump it up in sight again if nothing else.

if this act is not kosher, my apologies in advance.

There are simply two threads, this one, and the longer one you referred to

http://www.opengl.org/discussion_boards/cgi_directory/ultimatebb.cgi?ubb=get_topic;f=3;t=012842
http://www.opengl.org/discussion_boards/cgi_directory/ultimatebb.cgi?ubb=get_topic;f=3;t=012454

While fpo’s earlier thread was interesting because it more accurately resolved the new surface sample point than parallax mapping (through dependent read itteration to compute the ray-heightfield intersection) it did not handle silhouettes correctly.

His new algorithm improves on this and appears to treat silhouettes correctly so it’s even more interesting.

I’ve locked the other thread because I want to avoid confusion, bumping the old thread is fine but people will get confused with mistaken cross references and misleading questions w.r.t. the techniques so I’ve deleted one new post in the old thread to keep the distinction clear.

Please check post dates before jumping to conclusions and have fun.

fpo, awesome work again, I can’t wait for the demo.

Hi… as it seams we have some people interested in the new demo here it is. It is not perfect and does have problems specially at silhouettes when depth it too large (sampling problems). Also I represent surface curvature by a simple quadric at each vertex in two directions only (tangent space directions) so it is a huge approximation for general surfaces. Works good with simple geometry like a cylinder but even on a sphere will get you bad artifacts on top/bottom where mapping/triaingles collapses to a single point.

I didn’t post the this demo beofore because it was running too slow. But recently I tried the nVidia shader performance tool (nvshaderperf) and I must say it helped me optimizing the code. it was doing 10 Mpixles/sec before but now it is doing 45 Mpixels/sec (much better) and runs full screen at 85 fps with relief object taking all screen 800x600.

I have also optimized the other locally planar relief mapping implementation using tangent space (from my SIGGRAPH I3D 2005 paper) and it is now doing 85 Mpilxes/sec (quite fast).

Optimized locally planar tangent space relief mapping demo and final I3D2005 paper at:

Locally planar relief mapping with depth factor
(aproximate self-shadows and depth correction, no silhouettes)

Locally planar relief mapping with depth in object space
(exact self-shadows and depth correction, no silhouettes)

Optimized curved relief mapping demo at:

Curved relief mapping with depth in object space
(exact self-shadows and depth correction, good silhouettes)

The first demo uses depth aspect so depth=0.1 will mean 10% of the tile width in scence space. Here is mapping expands/compresses through object surface apparent depth will also change. This version does not require any extra vertex data (just regular position and texcoord).

The other two demos uses object space depth so depth=5 will mean apparent depth of 5 scene units. This requires extra data in vertices. The planar one needs mapping scale information stored as 2 extra floats per vertex telling size of texture tile at each vertex). The curved one needs 4 extra floats per vertex as it also requires curvature information at each vertex (2 floats).

Enjoy, FPO

Originally posted by Volker:

but whats the performance?
compared to parallax mapping and other techniques?

I used nvshaderperf to check the speed (pixel rate) of each shader. A Parallax mapping implementation gave me 220 Mpixels/sec and the locally planar relief mapping 85 Mpixels/sec. The curved relief mapping (with better silhouettes) is doing 45 Mpixels/sec. I’m sure more optimizations are possible but requires more time doing variations and checking performance.

Originally posted by Volker:

also compared to a displaced high geometry version of the object?

Forget about displacing a high resolution geomerty… I tested that and requires many million vertices and still looks much worse than the relief mapped version. The per-pixel displace is much better and texture fitering makes smooth trasitions from depth to depth… and triangulationg adds terrible artifacts to the displaced geometry. Trust me on that.

Originally posted by Volker:

did you test it with more arbitrary meshes?
how about inwards looking meshes (eg cornell box), how well does it perform at the seams?

Curvature is stored at each vertex with a quadric and interpolated through fragments. This is not perfect and surfaces with complex varing curvature will not work good. But depth correction is perfect for interpentarting surfaces like connection from two walls in a cornell box for example… no problem with that… you will see bricks intersecting correcly at the wall edges as you would expect from real geometry.