I’m working on a 3D geographical data renderer with building models on a terrain surface. These building models are captured through photogrammetry, and a problem we have is that the terrain surface sometimes pokes through the building model since the surface data and building model don’t match exactly.
We want to mask away the terrain surface in the area that is covered by the building model footprint. I’ve been thinking of using the stencil buffer, maybe extruding some kind of shadow volume from the model and filling the z buffer with high values in the area covered by the building model’s footprint before rendering the model. This would require quite a bit of processing though, and I’m hoping that there is smarter and more efficient way of doing things. Another idea is making an orthographic 2d texture of the model rendered from above and using this to fill the z-buffer in some creative way using shaders.
So if anyone have done something similar before or have any ideas, I’d be real glad to hear them
I’m limited to OpenGL ES 3.0, so I can’t use geometry shaders or other fancy features.
By “footprint”, do you mean a 2D shape in the horizontal plane? If so, you can use the fragment shader to discard fragments based upon looking up their world-space coordinates in a “plan” texture. You can use a signed distance map (where the texture value is the distance inside or outside the footprint) to improve accuracy.
If you want to clip out terrain using convex 3D volumes, stenciling is the correct approach. If you render the terrain, then render the volumes into the stencil buffer (colour writes disabled, depth writes enabled, depth tests enabled, stencil tests disabled, and a stencil op of GL_INVERT for depth pass and GL_KEEP for depth fail), any non-zero values in the stencil buffer correspond to fragments where the terrain is inside a volume. Rendering a closed volume should result in each fragment being rendered an even number of times (every pixel covered by a front face is also covered by a back face). An odd stencil value indicates that the front face was rendered but the back face failed the depth test. You can then remove the corresponding terrain by rendering the back faces of the volumes with stencil tests enabled, depth tests disabled, depth writes enabled. Thereafter, rendering geometry which is inside the volume will encounter depth values which are either in front of the volume or behind it, but not inside it.
As you can see there is quite a lot of terrain in the 3D model too, which may conflict with the underlying terrain model.
I consider the footprint as the vertical projection of this model onto the terrain. It’s not exactly a 2D shape, but it could be represented as a 2D texture if I rendered the model from above into a texture with an othographic projection.
The terrain itself is rendered in a geocentric coordinate system, but I guess I could send in the orthographic transform I used when creating the footprint texture as a uniform and use this to project the incoming geometry and then discard the fragments in the shader if they are inside the texture? Is this what you had in mind for the first method?
For a stencil volume I guess I would have to extract the outer perimeter of the 3D model and extrude it as a volume. I’m using a similar method to render polygons on the terrain, so I think I could make this work.
The first method seems a bit simpler and quicker though, if I have understood it correctly.
I think maybe I have a solution: For each tile in the terrain I render all the intersecting models into a texture with an orthographic viewport that matches the tile borders. This should give a texture that matches the tile with all the model footprints in it. Using this texture in the terrain shader, I can simply discard all fragments where this texture is different from 0.
We ended up using a stencil technique with a separate geometry for terrain occluders that is usually a bit smaller than the model outline, so that the terrain partially overlaps with the terrain geometry, avoiding possible gaps.
Thanks for your input Cameni, I’ve been following your blog articles on Outerra for many years. Lots of excellent stuff!
Anyway, I implemented it as described above, by rendering each model into a texture for each terrain tile that intersects the model. This footprint texture is sampled in the fragment shader and if it has value, I simply push the depth value of the corresponding fragment a bit back. By adjusting the depth value instead of discarding the fragment altogether, I get a reasonable color value instead of blank pixels along the border of the model where it doesn’t match the terrain surface perfectly.
You could also use a smaller reference mesh here to avoid trouble along the edges, but in practice it works nicely if you sample the footprint texture with linear sampling and you test against alpha == 1 instead of > 0. Then you should only hit fragments that are inside the model footprint due to the edge sampling.
So this method seems to work as it should, and it has the advantage that you only need to render the model once into each tile, instead of having to render the reference mesh into the stencil buffer for each frame. It requires some sort of tiled terrain model though.
Please shout if you see any stupid weaknesses with this method that I might have overlooked
