Weird behaviour with mod() and texture sampling

OpenGL
1

When drawing a 2D scene (ortho) I have a bunch of quads that read from one texture atlas to get their texture, I calculate the uv coordinates using this code:

float x = (uv.x * (_atlasRect.y - _atlasRect.x) + _atlasRect.x) / atlasSize.x;
float y = (uv.y * (_atlasRect.w - _atlasRect.z) + _atlasRect.z) / atlasSize.y;

This works perfectly, however, I want to also tile my textures using a vec2 _scaleFactor. This is the number of times it should repeat. For horizontally stretched objects it is equal to (scale.x/scale.y, 1.0f) and for vertically stretched objects it is equal to (1.0f, scale.y/scale.x). I multiply the uv by the _scaleFactor and then mod it by 1.0 in the fragment shader:

vec2 uv = _uv * _scaleFactor;
uv.x = mod(uv.x, 1.0);
uv.y = mod(uv.y, 1.0);

Which makes the uv look like this (before the atlas calculation).

This is great and works perfectly. Then it is scaled by the atlas calculation, which also works and gives an max x UV of 0.5 for the player (the atlas is 256x128 currently) with a min x uv of 0.5 for the ground, so this also works.

Then when I finally put the atlas UV coords into a texture sampler:

color = texture(tex,  vec2(x, y));

It has a black artefact around the texture.

It appears on all sides on the player (droplet) with a scale factor of (1.000, 1.000) and only on top and bottom of the platform (dirt) with a scale factor of (6.000, 1.000), and dissapears at some x positions.

  • If I don’t use mod() the artefact goes away (although the textures are wrong).
  • If I don’t multiply by scale factor and don’t mod() there’s no artefact.
  • If I use uv.x - fract(uv.x) (any uv.y) instead of mod(uv.x, 1.0) the artefact only appears.
  • If I use uv.x - floor(uv.x) the artefact still appears.

I tried rewriting the mod function to use doubles, no change. I tried changing the texture wrap parameters from clamp to repeat, no change. I have tried clamping from 0.0 to 1.0, no change. The UVs look perfect when I output them as the colour, so how can this thin border appear when sampling the texture? It doesn’t appear to be a pixel from anywhere in the texture.

I have tried casting the floored coord to a float, no change. I have tried multiplying by scale factor in the vertex shader instead, no change. The artefact only appears when using mod() on the uv, or perhaps floor().
Using GTX 1080Ti driver 560.94 OpenGL core 4.5 (although the issue persists when using opengl 3.2 in the shader with #version) with SDL2. The scale factor, matrix and atlas rect are passed in via an instanced array buffer, the atlasSize is a uniform. All the sprites are drawn with one call to glDrawArraysInstanced(). The instanced array buffer is passed in properly (verified through nsight graphics 2022.4.1).Transparency is enabled with glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);. Face culling is disabled.

Everything was working perfectly until I tried to mod the uv coord.

Vertex Shader code:

#version 450 core

layout(location = 0) in vec3 position;
layout(location = 1) in vec2 uv;
layout(location = 2) in mat4 pvmMatrix;
layout(location = 6) in vec4 atlasRect;
layout(location = 7) in vec2 scaleFactor;

out vec2 _uv;
out vec4 _atlasRect;
out vec2 _scaleFactor;

void main()
{
gl_Position = pvmMatrix * vec4(position, 1.0);
_uv = uv;
_atlasRect = atlasRect;
_scaleFactor = scaleFactor;
}

Fragment Shader Code:


#version 450 core

in vec2 _uv;
in vec4 _atlasRect;
in vec2 _scaleFactor;
out vec4 color;

uniform sampler2D tex;
uniform vec2 atlasSize;

void main()
{
//convert 0-1 uv coords to coords relative to atlas
vec2 uv = _uv * _scaleFactor;
uv.x = mod(uv.x, 1.0);
uv.y = mod(uv.y, 1.0);
float x = (uv.x * (_atlasRect.y - _atlasRect.x) + _atlasRect.x) / atlasSize.x;
float y = (uv.y * (_atlasRect.w - _atlasRect.z) + _atlasRect.z) / atlasSize.y;
color = texture(tex,  vec2(x, y));
}

Texture Creation:

glActiveTexture(GL_TEXTURE0); //select the default texture unit (to avoid messing up another texture unit's texture)
glGenTextures(1, &texture); //gen empty tex
glBindTexture(GL_TEXTURE_2D, texture); //bind it
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); //set wrapping values
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); //..
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST_MIPMAP_LINEAR); //set filter values
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); //..
glTexImage2D(GL_TEXTURE_2D, 0, glType, width, height, 0, glType, GL_UNSIGNED_BYTE, data); //populate texture
glGenerateMipmap(GL_TEXTURE_2D); //generate mipmap

Originally I had these embedded seperately (not allowed), then as imgur links (not allowed) so here are my images all as one:

oglforum
oglforum1920×1291 87.5 KB

2

This is a common problem. texture() needs sensible texcoord derivatives (for uv in your case) to infer how fast the texcoords are changing across each 2x2 group of pixels. It does this by computing a difference in the texcoord values across adjacent pixels. This rate-of-change in the texcoord is then used to determine which texture MIPmap level(s) to sample. Small changes = sample the base map (or finer resolution MIPmaps). Large change = sample the courser, lower-resolution MIPmaps.

The problem comes in when those derivatives are either huge or garbage. This tells the GPU there’s a huge change in the texture coordinates in 1 pixel, so it needs to jump way up in the texture MIPmap stack and sample from one of the courser, lower-resolution, minified MIPmaps for those pixels. This results in an artifact where these huge jumps in the texcoord occur.

To see if this is related to your problem, try:

  • Set GL_CLAMP_TO_EDGE for your texture wrap modes.
  • Set your MIN_FILTER to GL_NEAREST or GL_LINEAR.
  • Disable anisotropic filtering on your texture.

If the lines go away, then they were related to texture sampling issues due to bad texcoord derivatives, or (less likely) pulling in border texel colors.

If this is the problem, you can compute explicit texcoord derivs before the mod() operation and pass that into a GLSL texture sampling function that accepts these derivs.

1 Like