Manual bilinear filter


vec4 texture2D_bilinear(in sampler2D t, in vec2 uv, in vec2 textureSize, in vec2 texelSize)
{
    vec4 tl = texture2D(t, uv);
    vec4 tr = texture2D(t, uv + vec2(texelSize.x, 0.0));
    vec4 bl = texture2D(t, uv + vec2(0.0, texelSize.y));
    vec4 br = texture2D(t, uv + vec2(texelSize.x, texelSize.y));
    vec2 f = fract( uv * textureSize );
    vec4 tA = mix( tl, tr, f.x );
    vec4 tB = mix( bl, br, f.x );
    return mix( tA, tB, f.y );
}

I’m using this in a vertex shader, sampling from a non-mipped GL_NEAREST filtered texture. It works fine for most textures, but as the uv input parameter gets bigger (or the texture size gets smaller, not sure which), it breaks down (e.g. random stepping in the mesh).
Fair enough, I can live with something that could be a precision error.
BUT, it gives different results to sampling directly using texture2D from a texture with bilinear sampling enabled on it - which works fine, no matter what my uv or texture size is.
My question is, is my bilinear sampling function wrong? Is there another more correct way?
thanks.
pete.

For me your bilinear filter looks correct. Be sure you calculate textureSize & texelSize correctly.

You can post more code and add some screenshots to make the investigation easier.

Thanks for your answer. I guessed it was ok, it makes sense to me, plus I’ve seen lots of sample code on the web that’s the same.
I’m 100% sure the textureSize is correct, and texelSize is calculated as 1.0/textureSize, which I assume is correct…unless I’m missing some border issue you all know about.
I can’t really post a screenshot - the only data I’ve got is quite ‘sensitive’, and I’d get into trouble posting it on a public forum.
The uv could possibly be quite a mad number, as it’s the output of some convoluted logic. It shouldn’t matter that it’s a mad number really, because the texture has GL_REPEAT texcoord addressing enabled. But, as I say, it works for most texture sizes/uv configurations, and also the built in bilinear sample doesn’t have a problem with them. Simply replacing the body of that function with a “return texture2D(t, uv);” with GL_LINEAR filtering enabled on the texture gives correct results every time. I need this manual method as a fallback for cards that don’t support bilinear filtering in a vertex shader.
thanks for listening, anyway.
love and kisses,
pete.

…edit…
BTW, I’ve tried replacing the texture2D’s with texture2DLod’s, because I realise the vertex shader has no means of determining the correct mipmap to use (even though the texture has no mips, nor does it have a mip filter enabled). Made no difference at all.

if the uv coordinate were so large that, when multiplied by the textureSize, it became a huge number, then I’d lose some needed precision in the fractional part, wouldn’t I?
So when I take the fractional part, I’d be getting some truncated number.
Could this be the root of my problem?
If I used mod(uv, 1.0) would that deal with negative uv values?
say, if I did this:
vec2 uv(-10.0, -10.0);
uv = mod(uv, 1.0);
what would be the new value of uv?

From the spec:


genType floor (genType x): Returns a value equal to the nearest integer that is less than or equal to x.

genType fract (genType x): Returns x – floor (x).

genType mod (genType x, float y)
genType mod (genType x, genType y)
Modulus. Returns x – y * floor (x/y).

So, mod(-10.0,1.0) will give you 0.0

About the precision after multiplication:
Assuming you have ~1000 (2^10) texture size.
Float32 has 23 bits mantissa. The multiplication result will have not more than 10 bits before the dot (what you don’t need) and not less than 13 bits of precision you need. This is more than enough for determining bilinear coefficients.

P.S. I have no idea what’s wrong ATM

Yes, you’re correct, thanks.
Tried it anyway, just as a sanity check, and no difference. I still have the problem. It is bizarre.

Well, fixed it.
Not happy with how though.


vec4 texture2D_bilinear(in sampler2D t, in vec2 uv, in vec2 textureSize, in vec2 texelSize)
{
    vec2 f = fract( uv * textureSize );
    uv += texelSize*0.06;    // <--- precision hack (anything higher breaks it)
    vec4 tl = texture2D(t, uv);
    vec4 tr = texture2D(t, uv + vec2(texelSize.x, 0.0));
    vec4 bl = texture2D(t, uv + vec2(0.0, texelSize.y));
    vec4 br = texture2D(t, uv + vec2(texelSize.x, texelSize.y));
    vec4 tA = mix( tl, tr, f.x );
    vec4 tB = mix( bl, br, f.x );
    return mix( tA, tB, f.y );
}

Ugly, but it works in all cases now.
It was obviously point sampling the same texels occasionally, so the slope was lost.
Anyone shed any light on why?

NVidia Quadro 5800, driver: 190.38 OS: XP64,SP2

Feel a bit silly reviving such an old thread, but I’ve just had the same problem, know what the problem is and have a correct solution.

The artifacts are caused as the fraction approaches 1 and numerical inaccuracies causes it to wrap around to 0. This causes the mix operation to generate a hard step and the artifact.
Power of 2 textures help as the calculations become little more than bit shifts, meaning less chance of errors.

Fixing things requires making the error in the fraction irrelevant. To do this uv is moved to the texel centre using the result from fract, that way if its suppose to be .9999 but returns 0, uv, is positioned correctly.
As for choosing the texel centre rather than the texel origin, again, this is due to numerical inaccuracies sometimes causing the wrong texel to be picked, where as, nearly the centre is always good enough. As a bonus it doesn’t matter if the filter is GL_NEAREST or GL_LINEAR.

vec4 texture2D_bilinear(in sampler2D t, in vec2 uv, in vec2 textureSize, in vec2 texelSize)
{
    vec2 f = fract( uv * textureSize );
    uv += ( .5 - f ) * texelSize;    // move uv to texel centre
    vec4 tl = texture2D(t, uv);
    vec4 tr = texture2D(t, uv + vec2(texelSize.x, 0.0));
    vec4 bl = texture2D(t, uv + vec2(0.0, texelSize.y));
    vec4 br = texture2D(t, uv + vec2(texelSize.x, texelSize.y));
    vec4 tA = mix( tl, tr, f.x );
    vec4 tB = mix( bl, br, f.x );
    return mix( tA, tB, f.y );
}
1 Like