Texture atlas bleeding issue on 3D sprites w/ GL_NEAREST

OpenGL OpenGL: Basic Coding
1

For an OpenGL 3.1 app, I’ve texture atlas’d all of my decorative sprites which sit on my terrain, into one sprite sheet, as there are lots of them and I’d prefer to avoid lots of texture binds. Also, I’m going for an intentional retro look which means GL_NEAREST is fine for me. All looks good when moving the camera/player around – mostly, that is. When the camera is at certain positions/angles, textures on sprites may bleed into the neighboring sprite, even on GL_NEAREST. These errors pop in and out, and aren’t always there. For example, if I stop the camera at such a position that a bleeding error appears on the tree sprite in front of me, and if I then move my mouse 1 pixel in any direction, that artifact may vanish again.

859×747 229 KB

I’m not convinced that inserting padding pixels around every single sprite is the best solution because then I’d kind of be destroying one of the points of a texture atlas - to have efficiently and tightly packed textures. Putting 1-2px border around every sprite would seem to be wasted space, would reduce the available graphical detail of every sprite, (12-14px instead of 16px wide), and (I think) theoretically it may not solve the problem anyway if I move too far away into the distance.

“Fudging” the UVs to render in the frag shader also seems wrong to me. i.e. Rendering 0.0 + 0.0001 to 0.125 - 0.0001 rather than 0.0 to 0.125, as we will then be squashing/deforming the shapes of the pixels at the edge of every sprite.

How should I otherwise solve this problem, if even possible, without going the “different texture for every single sprite and use CLAMP” route?

My textures are set up like this:

    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
    glTexParameterI(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);
    glTexParameterI(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
    glTexParameterI(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);

And in the fragment shader, I retrieve the UVs like:

    in vec2 fUVs;
    uniform sampler2D uTex0;
    ...
    vec4 tex_color = texture(uTex0, fUVs);

And when creating those UVs on the CPU, I’m doing basic float math such that the UVs values (should?) always be able to be represented accurately by floating points, since all sprites are either powers of 2, or at least, multiples of 16.

Also, I’m not (I don’t think) using mipmaps. Should I be? Or would that make bleeding even more likely, in this case?

2

From your texture filtering, it looks like you’re not using MIPmaps. In that case, I would consider:

  1. Making sure that anisotropic filtering is disabled for your textures, and then (if necessary)
  2. Insetting your texcoords for each texture in the atlas by 1/2 texel on the edges.

I believe that should avoid both your wrap behavior from kicking in as well as sometimes sampling texels from the adjacent texture in the atlas (as you will when you let texcoords run all the way to the edges of the outer ring of texels of a texture in the atlas).

3

Thank you for your reply,
What you describe about insetting the texcoords is what I meant by fudging the UVs

827×606 39.6 KB

Unless I’m misunderstanding you, this results in “squashed” pixels along the sprite borders, you can see the colored pixels are noticeably thinner than the rest of the sprite pixels which is not an effect that I want. (Of course, I could simply reduce the amount I’m cutting off, but then the squashing is still there, just lessened.)

As for ensuring anisotropic is disabled, as far as I know, I only have GL_NEAREST filter on everything, or are you referring to something else?

Is there another way to solve this? I would even be happy with some kind of hard-coded cheesy brute-force check in the frag shader which can detect, “HEY, that pixel you accessed just went out of bounds of that specified UV area! Discard it instead.” or something, but I’m unsure how I’d do that.

4

With GL_NEAREST and texture coordinates on 1/2^n boundaries, you shouldn’t be getting these artefacts. However: It’s impossible to get texture coordinates exactly on 1/2^n boundaries if you use a normalised format because the divisor is always 2^n-1. So you either have to use a floating-point attribute array or “fix” the coordinates in the shader.

For the latter, multiply by the appropriate scale factor so that the resulting texture coordinates should be integers, round to the nearest integer, then divide by 2^n. E.g. if the sprites are 16x16 within a 64x64 texture, multiply by 4, round(), then divide by 4.

You probably shouldn’t be. But if you do use mipmaps, you need to set the maximum mipmap level (GL_TEXTURE_MAX_LEVEL) such that the individual sprites within an atlas are never smaller than a texel.

5

Firstly thank you for the response -

"With GL_NEAREST and texture coordinates on 1/2^n boundaries, you shouldn’t be getting these artefacts." Hmm, what does that tell me then since I was getting them intermittently? The coords were definitely 1/2^n, or at least, binary friendly, like 0.25 or 0.75 where there still wouldn’t be any floating point error AFAIK. ?

"However: It’s impossible to get texture coordinates exactly on 1/2^n boundaries if you use a normalised format because the divisor is always 2^n-1." Ahh, ok, well, I do calculate the UVs of those sprites one time when the terrain is first generated, and they are indeed normalized (erm, normalized in this case means between 0.0 and 1.0, right? Haha…just making sure.). Should I, instead, simply save the UV’s at giant whole integers, rather than 0.125, 0.625, and such, and make less work for the shader?

" For the latter, multiply by the appropriate scale factor so that the resulting texture coordinates should be integers, round to the nearest integer, then divide by 2^n. E.g. if the sprites are 16x16 within a 64x64 texture, multiply by 4, round() , then divide by 4." OK, a couple things here. Firstly, when I tried this, the pixels were gigantic. I then realized why – I believe you meant that the scale factor would need to be how many pixels across in the entire sprite sheet, rather than how many tiles across. Right? Because if I just go by tiles, i.e. let’s use your example of four 16px tiles across with 64x64px total, if I multiply any UVs in the frag shader by 4, for let’s say the first tile of 0.0 to 0.25, then the values would end up being 0 or 1 after being rounded, which just results in one giant same-colored pixel. I also needed to offset by a half-pixel correction, similar to what Photon suggested earlier, and I assume that’s because of how Round() behaves. Having said that – and if I have not indeed misunderstood your instructions – I nonetheless understood your concept, and I multiplied everything by the pixel span instead of the sprite span (in my case, the sprite sheet is 256x256, so I multipled by 256), and then I used Floor() rather than Round(), divided by 256.
Somehow, in doing Floor rather than Round, I was able to not need the half-pixel shift I’d just mentioned, which was nice. I was a little scared about using Floor() because I thought it might shave off the very last pixel of every sprite, or something, but it doesn’t seem to be doing so. Anyway, all of this information is basically to ask you if my adjustments were correct, or did I misunderstand your advice? (Either way, I’ve already learned much from it, but I’m just curious!)

For reference, here is the relevant frag shader part:
in vec2 fUV;
...
void main()
{
vec2 new_uv = fUV;
new_uv.x = (new_uv.x) * 256;
new_uv.y = (new_uv.y) * 256;
new_uv.x = floor(new_uv.x)
new_uv.y = floor(new_uv.y)
new_uv.x /= 256
new_uv.y /= 256
vec4 tex_color = texture(uTex0, new_uv);

One final thing. This solution definitely worked, but the artifact still happens very, very, very occasionally. Magnitudes less frequently than before. The reason I dare to even mention this, is just in case that anything I’ve said/asked in this post here might possibly explain why it might still be occurring very occasionally, or how to eliminate it (for example, passing in non-normalized UVs or something?). So I’m very happy with it, but am still curious about the previous things mentioned.

6

In this case, “normalised” means calling glVertexAttribPointer with the normalized parameter non-zero and the type parameter indicating an integer type (something other than GL_HALF_FLOAT, GL_FLOAT or GL_DOUBLE). It’s fairly common to use 8-bit or 16-bit integers for texture coordinates as you rarely need the precision offered by floats.

Unsigned normalised types cover the range 0…1 inclusive. Values are converted to floats by dividing by 2^n-1 where n is the number of bits in the type. Because of this, values which are exact binary fractions (i.e. x/2^n where x and n are integers) aren’t exactly representable as normalised values.

No, I meant tiles. This assumes that you only ever use texture coordinates which are (supposed to be) “grid points”, i.e. the corners of sprites/tiles within the atlas.

The idea is that the values will be integers (or almost integers) after the multiply. round() forces them to be exact integers (correcting for any rounding errors which may exist in the texture coordinates), then the division “undoes” the multiply, converting them back to normalised texture coordinates (i.e. in the 0…1 range, different use of “normalised” to before).

A half-pixel offset is usually the wrong thing to do with GL_NEAREST. You might want a very small offset (like, 1/100th pixel or so) to avoid artefacts in the case of rendering a sprite which is offset from the window’s pixel grid by exactly half a pixel (meaning that the texture gets sampled exactly on the edge of a tile).

Finally: if all of the sprites are the same size, you could avoid these issues by using an array texture rather than an atlas. An array texture is somewhat similar to a 3D texture, but no filtering or wrapping is performed in the third dimension, and the third texture coordinate isn’t normalised (it’s just rounded to the nearest integer to obtain the layer number). So the s,t coordinates will always form a unit square and the third coordinate selects the sprite.

7

Thank you so much. Been going through some things, but got back to this today. I passed in the uvs as unsigned chars instead (considering the low res of my game, it works lol), and then do math in the shader. I understand now about working with whatever the lowest size of a tile/sprite is and treating that as intervals of 1. Because of course you’re right, don’t need the precision of floats here.

Small note, round() still didn’t work for me. When I use round(), it shifts things by a 0.5 texel offset. Floor() worked fine for me though. Even though I had only originally asked about sprite UV bleed, I also had minor bleeding on my terrain if you looked real close, since the different blended terrain textures (grass, dirt) were atlas’d as well. Those, too, have now been fixed. Thanks so much. :slight_smile: