My goal is to have various rectangular patches on my ground, which can be dug up, revealing a ditch below the ground. Currently, I have the ground drawn as one large quad, and it blocks everything under it (looking from above). From what I have researched, there is no convenient way to erase part of a drawing in OpenGL. Note that the positions of these patches are accessible (and drawn as quads), so this would be a really convenient way.
My second idea is to never draw those parts, instead of erasing them. Is there any way to make a call to draw ignore certain coordinates? Can I establish a sort of drawing dominance that prioritizes the ditches (the bottom of the ditch and its walls)?
In the worst case scenario I would have to draw convex polygons that surround the patches and include their vertices, but it would be great if OpenGL had a way to do that already. It would be really difficult to do for a randomly generated map, unless someone could suggest a good algorithm. Thanks!
You can use stencilling to prevent specific pixels from being drawn to.
Stencil operations such as increment, decrement and invert can be used to implement constructive solid geometry (CSG) operations, where you render the intersection, union or difference between shapes.
One way to implement your particular case would be:
[ol]
[li] Render the ground quad into the depth buffer.
[/li][li] Clear the stencil buffer.
[/li][li] Enable stencil tests.
[/li][li] Enable depth tests.
[/li][li] Disable depth writes.
[/li][li] glStencilFunc(GL_ALWAYS, 0, 0)
[/li][li] glStencilOp(GL_KEEP, GL_KEEP, GL_INVERT).
[/li][li] Render a closed volume to be “cut out” from the ground to form the ditch, with both front and back faces rendered.
[/li][li] glStencilFunc(GL_EQUAL, 0, 1)
[/li][li] Render the ground quad into the colour buffer.
[/li][li] Disable depth tests.
[/li][li] Enable depth writes.
[/li][li] glStencilFunc(GL_NOTEQUAL, 0, 1)
[/li][li] Render the cut-out volume into the colour buffer to form the ditch.
[/li][/ol]
How this works is that, for portions of the volume which aren’t clipped, every pixel will be rendered twice (once for a front face, once for a back face), resulting in the stencil buffer containing zeros (the initial value will be inverted twice). Any pixels corresponding to the portion of the ground inside the volume will obscure that pixel in a back face but won’t obscure the pixel in the front face, resulting in the stencil buffer containing ones (the initial value will only be inverted once). When the ground is rendered, the pixels inside the cut-out will be discarded due to the stencil test. When the cut-out volume is rendered, the pixels above the ground will be discarded due to the stencil test.
However, it may be simpler to just tessellate the ground into multiple polygons so that you can just skip rendering specific polygons.
CSG operations are more useful when the shapes (and thus their intersections) can change dynamically, meaning that the results cannot be pre-computed.