Blend function w/ both source and dest alphas?

So I know that glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) will give me a function much like this:

vec4 over(vec4 a, vec4 b) { // a is over b
    vec4 result = a * a.a + b * (1.0 - a.a);
    result.a = 1 - (1.0 - a.a) * (1.0 - b.b);
    return result;
}

But this won’t work for me. I need a function that will do this:

vec4 over(vec4 a, vec4 b) { // a is over b
    vec4 result = a * a.a + b * b.a * (1.0 - a.a);
    result.a = 1.0 - (1.0 - a.a) * (1.0 - b.a);
    return result; 
}

(notice the “* b.a” in the first line)

which is more consistent with the standard “over” operation, because it takes into account both the source and destination alphas. The former only takes into account the source alpha, probably because most render targets don’t have destination alpha. (My destination does have an alpha channel, because my render target happens to be used as an overlay in my shader)

I wish there was a GL_DST_ALPHA_AND_ONE_MINUS_SRC_ALPHA, so I could do what I need.

Please help me, how can I get blending that does a standard “over” calculation?

But this won’t work for me. I need a function that will do this:

Why? What does this equation mean?

Please help me, how can I get blending that does a standard “over” calculation?

I don’t know what you mean by “standard over”, but the equation you want is not possible with OpenGL blending. If you need this, then you’re going to have to do programmatic blending. That means rendering something to a texture, then using the texture as input in the rendering of something else. For each thing you need to use this “standard over” for.

My bad! I meant to include a link to explain what I meant by “over”. It’s at http://en.wikipedia.org/wiki/Alpha_compositing, at the end of the Description section, it has this img: where Co is the final color, Ca is the src color, and Cb is the dest color.

Tis a shame opengl can’t do this, it seems a pretty basic thing for them not to include. So I can imagine a rough way to do this with different textures, but it would require reading the destination alpha from my target texture, and then writing the new color to that target texture, which is concurrent read/write. Are there any resources you can point me to that explain how to do this?

Thanks!

The equation you quoted is correct if you are only looking at two objects being blended together. When you have layers upon layers, you have to composite them one at a time. In doing so, each layer’s color gets its own alpha multiplied into itself (Ca * alpha_a). So the destination color already had its alpha factored in; there’s no need to do it twice.

Just try the following:

glBlendFuncSeparate( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE, GL_ONE_MINUS_SRC_ALPHA );

It should result in a quite good blending equation.