Floating point framebuffer and multisampling

OpenGL OpenGL: Advanced Coding
1

Hello. I have a multisampled framebuffer with two color attachments, one with RGBA8 and another with R32F and I want to blit it to another framebuffer, to resolve the multisampling (, do some other work) and finally render the final image.

After I blit the first framebuffer to the second, I want to read from the second framebuffer with glReadPixels, to implement mouse picking, so read one pixel, one specific float value. The problem is that when I hover my mouse on and read from any edge of any model (that is anti-aliased), the value read is something apparently random, not the the correct one.

I disabled multisampling and tried the same thing and everything works fine, the float values read are always correct. This tells me that when I blit the framebuffer and I resolve both attachments, the float values at the edges get transformed in some way, which is not what I want.

My question is: is there some way to prevent this, or fix this problem? Can I do something?

If this is an anti-aliased model rendered on the screen, then the pipes represent the pixels with correct float values, and the colons represent the incorrect values:

:::::::::::::
:|||||||||||:
:|||||||||||:
:|||||||||||:
:|||||||||||:
:::::::::::::

Code for first framebuffer:

glGenFramebuffers(1, &framebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);

// First attachment
glGenRenderbuffers(1, &renderbuffer);
glBindRenderbuffer(GL_RENDERBUFFER, renderbuffer);
glRenderbufferStorageMultisample(
    GL_RENDERBUFFER, samples, RGBA8, width, height
);
glFramebufferRenderbuffer(
    GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0 + 0, GL_RENDERBUFFER, renderbuffer
);

// Second attachment
glGenRenderbuffers(1, &renderbuffer);
glBindRenderbuffer(GL_RENDERBUFFER, renderbuffer);
glRenderbufferStorageMultisample(
    GL_RENDERBUFFER, samples, R32F, width, height
);
glFramebufferRenderbuffer(
    GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0 + 1, GL_RENDERBUFFER, renderbuffer
);

// Depth and stencil...

GLenum attachments[2] = { GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1 };
glDrawBuffers(2, attachments);

Code for second framebuffer:

glGenFramebuffers(1, &framebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);

// First attachment
glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_BORDER);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_BORDER);
glTexStorage2D(GL_TEXTURE_2D, 1, RGBA8, width, height);
glFramebufferTexture2D(
    GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0 + 0, GL_TEXTURE_2D, texture, 0
);

// Second attachment
glGenRenderbuffers(1, &renderbuffer);
glBindRenderbuffer(GL_RENDERBUFFER, renderbuffer);
glRenderbufferStorage(
    GL_RENDERBUFFER, R32F, width, height
);
glFramebufferRenderbuffer(
    GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0 + 1, GL_RENDERBUFFER, renderbuffer
);

GLenum attachments[2] = { GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1 };
glDrawBuffers(2, attachments);

Code for blitting:

glBindFramebuffer(GL_READ_FRAMEBUFFER, read_framebuffer);
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, draw_framebuffer);

for (size_t i = 0; i < 2; i++) {
    glReadBuffer(GL_COLOR_ATTACHMENT0 + i);
    glDrawBuffer(GL_COLOR_ATTACHMENT0 + i);
    glBlitFramebuffer(0, 0, width, height, 0, 0, width, height, GL_COLOR_BUFFER_BIT, GL_NEAREST);
}

glDrawBuffers(2, attachments);

All rendering looks somewhat like this:

// Bind the first, multisampled framebuffer

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
glViewport(0, 0, width, height);

// Disable blending to not mess up the second, floating-point attachment
glDisablei(GL_BLEND, 1); 

// Render all models...

// Re-enable blending
glEnablei(GL_BLEND, 1);

// Blit the resulted scene texture to an intermediate texture, resolving anti-aliasing

// Bind the second, normal framebuffer

// Read one pixel with glReadPixels

// Here I can print and see the float value

// Render the final image to the screen

I can provide more details, if needed.

2

What would “the correct one” even be? The entire point of multisample anti-aliasing is to blend edges of primitives smoothly. If you need to do picking, then you need an exact value. So you shouldn’t be using anti-aliasing at all.

3

Well, I have a “scene” framebuffer in which I render various objects with and without mouse picking functionality, and I also need that framebuffer to be multisampled, so that the scene would look good. I cannot set one attachment with 4 samples and another with just one. And I wouldn’t prefer to render the objects twice, just to fill up a second, non-sampled float framebuffer.
I don’t know what would be the best solution for this problem.

4

Previously, I had the second attachment of type R32I, so an integer attachment. But on one of the platforms I use, the OpenGL implementation didn’t support integer multisampling at all (GL_MAX_INTEGER_SAMPLES = 1). It did support floating-point multisampling (GL_MAX_DEPTH_TEXTURE_SAMPLES, GL_MAX_COLOR_TEXTURE_SAMPLES).

But just because the max samples for integer formats was 1, I couldn’t use anti-aliasing at all and the program crashed.

GL_MAX_SAMPLES was returning 8, even if GL_MAX_INTEGER_SAMPLES was 1.

That’s why I switched to a float buffer type.

5

But you don’t have a choice either. You cannot do a read-pixels on a multisampled framebuffer at all, and multisample resolve will by its nature destroy whatever information you’re trying to use.

You are going to have to render the scene twice. Best to accept this.

OK, there is an alternative. While the CPU cannot read from a multisampled image, the GPU can. So you could launch a compute shader operation (after unbinding the FBO) that reads from a specific location in the multisampled image and writes what value it reads to some buffer.

Of course, you’ll have to figure out what to do if different samples in the texel have different values. That is, you’re going to have to figure out what it means if many objects share the same texel. But you can do it.

That being said, you said that the hardware you care about only has an integer sample count of 1. Most modern hardware has higher sample counts, so you’re working with something old that may not have compute shaders. You might be able to get a fragment shader to do the same thing, but that’d be harder.

6

Since yesterday I thought about it and I think I’ll just render separately some instanced bounding boxes around the objects, into a separate framebuffer and read from that.

This solution actually will help me a lot, because eventually I’ll want to port the application to Android too and with bounding boxes I’ll have much more flexibility regarding how an object can be selected with the mouse, or finger on Android. (On a phone screen the objects could be hard to pick, if they are too small.)

On the other hand, I could just give up on anti-aliasing on those 3.7% of the implementations, but I’ll choose the solution above anyway.

Curiously, when I had Ubuntu 18.04 on my laptop with Intel HD Graphics 4400 and some version of Mesa, it did support integer multisamping, but when I upgraded to Xubuntu 22.04.1, with Mesa 22.0.5, integer multisampling was not supported anymore.

Using compute shaders would be interesting, but would also complicate things a bit. And I haven’t actually used them anywhere yet.

7

The sensible solution would to be to ignore it (nothing picked). When I’ve used framebuffer-based picking, I normally read a small window around the cursor and only report a hit if there’s exactly one valid ID within that window.

With modern display resolutions, you can’t expect the user to position the cursor to the exact pixel. Particularly not with trackpads or touchscreens.

8

This could be a useful idea.

Okay. Thank you for the help!