How can I achieve a rotating sample pattern without super-sampling?

I want to slightly shift each sample (less than one pixel width) in a rotating pattern (like +0.25,+0.25; -0.25,+0.25; -0.25,-0.25; +0.25,-0.25 in pixel units, for 4 adjacent pixels). I’m not entirely sure how I could do that. One idea was to use 4x MSAA and to decrease the framebuffer resolution 4x at the same time (so that the effective resolution remains the same). However, I don’t want to downsample the resultig image (as I implied in the title), I’m not sure how I can choose the pattern and whether I need to take into account special settings of the FBO.

Ok.

One idea was to use 4x MSAA and to decrease the framebuffer resolution 4x at the same time (so that the effective resolution remains the same). However, I don’t want to downsample the resultig image (as I implied in the title), I’m not sure how I can choose the pattern and whether I need to take into account special settings of the FBO.

Hold up a second. You want a rotated grid sample pattern within each pixel, but you don’t want to downsample each pixel? Those two things don’t make sense together. What does that even mean?

Also re your title, it didn’t imply that you didn’t want to downsample. Are you saying that you don’t want to run the fragment shader for each sample (i.e. supersampling, aka SSAA)? – that’s what it says. Or are you saying you’d like to “simulate” rotated-grid MSAA, but without actually “using” the hardware’s MSAA or SSAA functionality?

If the latter, render several images with sub-pixel-offset frustums at the sub-pixel sample pattern that you want. Then composite them (i.e. implement your own downsample; there are a number of ways to do this; some trivial). Note that I think you can do the former all-in-one-render pass if you want.

[QUOTE=Dark Photon;1262300]Hold up a second. You want a rotated grid sample pattern within each pixel, but you don’t want to downsample each pixel? Those two things don’t make sense together. What does that even mean?
[/QUOTE]

That’s right, I just want each sample to be slightly shifted in the said pattern. The result will be filtered with a gaussian in a compositing fragment shader afterward.

Sorry for the confusion, I actually meant a 4x SSAA all-in-one-render pass. How can I achieve that? By simply setting the FBO, the color attachments and viewport size to w/2, h/2 and glEnable some hardware specific setting? I’m using NVIDIA, but I’m a little bit lost where to look for the specific commands and I couldn’t find a tutorial on SSAA. I would appreciate if someone could give me a pointer to a good resource how to set up the textures and the rotating pattern. Ideally I don’t want to change the texture at all apart from the dimensions of it.

Ok, so you “do” want to downsample. You just want to do it yourself, and not use the built-in downsampling functionality.

Sorry for the confusion, I actually meant a 4x SSAA all-in-one-render pass. How can I achieve that? By simply setting the FBO, the color attachments and viewport size to 2w, 2h and glEnable some hardware specific setting?

You could do that, but with 1x (no-FSAA) that doesn’t give you a rotated grid sampling pattern – that’s just a regular grid. And it’d be overkill to increase your res 8x*8x or something and just subsample that information. You could, but it’s gonna take a lot more memory.

And with the HW FSAA enabled, you can’t (AFAIK) control the sampling pattern within each pixel. You can query the sample positions, but I don’t think you can change them. So that doesn’t sound like what you want either.

I’m using NVIDIA, but I’m a little bit lost where to look for the specific commands and I couldn’t find a tutorial on SSAA.

You can read up on ARB_sample_shading, but it doesn’t sound like that’s really what you want. Sounds like you want a custom rotated-grid sampling pattern, and that only gives you the sampling pattern that the hardware has built-in. If you don’t want custom but you’re happy with the sample pattern that the driver already supports for some given mode (you can query the subsample positions), then you could just activate the mode and use that. If OTOH you do want a custom pattern, I described a method in the previous post that might work for you.

Ok, so you “do” want to downsample. You just want to do it yourself, and not use the built-in downsampling functionality.

I actually want to do the opposite. I want to upsample/blurr the image (even more).

What I basically want to do is to mitigate aliasing of this kind without increasing the sample count: http://i.imgur.com/3oLZLLcl.png, http://i.imgur.com/iFHtMs0.png. I think MSAA should work too as long the pattern is not just a regular grid. The idea is that the straight lines get dappled/dithered a little bit, so that there isn’t as much flickering in the final, blurred image.

Now I’m trying it with a code example from here. There are two problems however:
(1) The multisampling appears to use a regular grid because there is no improvement. (2) The second attachment is mistakenly the same as the first. Am I using the blitting commands correctly?



			...

			glGenTextures(1, &tex1);
			glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, tex1);
			glTexImage2DMultisample(GL_TEXTURE_2D_MULTISAMPLE, 4, GL_RGBA16F, w/2, h/2, true);
			glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D_MULTISAMPLE, tex1, 0);
			glGenTextures(1, &tex2);
			glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, tex2);
			glTexImage2DMultisample(GL_TEXTURE_2D_MULTISAMPLE, 4, GL_RGBA16F, w/2, h/2, true);
			glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D_MULTISAMPLE, tex2, 0);
			glDrawBuffers(2, {tex1, tex2});

			...

			m_multisampleFBO->bind();
			glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
			render();
			m_multisampleFBO->unbind();

			glBindFramebuffer(GL_DRAW_FRAMEBUFFER, m_FBO->getHandle());
			glBindFramebuffer(GL_READ_FRAMEBUFFER, m_multisampleFBO->getHandle());
			glDrawBuffer(GL_COLOR_ATTACHMENT0);
			glReadBuffer(GL_COLOR_ATTACHMENT0);
			glBlitFramebuffer(0, 0, w/2, h/2, 0, 0, w, h, GL_COLOR_BUFFER_BIT, GL_NEAREST);
			glDrawBuffer(GL_COLOR_ATTACHMENT1);
			glReadBuffer(GL_COLOR_ATTACHMENT1);
			glBlitFramebuffer(0, 0, w/2, h/2, 0, 0, w, h, GL_COLOR_BUFFER_BIT, GL_NEAREST);
			glDrawBuffer(GL_BACK);
			glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);
			glBindFramebuffer(GL_READ_FRAMEBUFFER, 0);


Edit: Oh, that can’t possibly work. I’m rendering a screen filling triangle, so there aren’t many edges. I do need SSAA. I would appreciate if someone could tell me how to activate SSAA.

It seems that the following commands enable SSAA:


                    glEnable(GL_SAMPLE_SHADING_ARB);
		    glEnable(GL_MULTISAMPLE);

There are however still two problems: Strangely enough, the supersampling is only enabled if I make use of the gl_SampleID in the fragment shader, for example by overriding the output of the second color attachment with vec4(gl_SampleID). The second problem is that it appears to be impossible to avoid the resolve step during blitting since glBlitFramebuffer does not allow for different dimensions of the rectangles:

        GL_INVALID_OPERATION is generated if GL_SAMPLE_BUFFERS for both read and draw buffers greater than
        zero and the dimensions of the source and destination rectangles is not identical.

Is there a different way to copy the multisample texture contents?

There is an NVIDIA extension for that called explicit multisampling (NV_explicit_multisample), explained here: http://28byteslater.com/2009/09/16/explicit-multisample-real-antialiasing-in-deferred-shading/

It works fine with that except I can’t get SSAA working for some reason. MSAA works fine (there are additional samples along the diagonal of the screen filling rectangle: http://i.imgur.com/glFaTxe.png).

Edit: It works now! http://i.imgur.com/wcKLxkV.png. The problem was that the GLSL compiler optimized the gl_SampleID * 0 away so it was disabled again. There must be a better way to enable SSAA than adding gl_SampleID * 0.000000001 though…