rendering an alpha key

hcatry · November 17, 2003, 4:57am

Hello,
I’m writing an opengl TV broadcast “game” that has some transparent elements in it. The game will be overlayed over the video by sending an alpha key map to the external video mixer. The alpha key is a grayscale rendering of the game scene using the alpha values of each pixel. I plan to render the alpha key map in a split screen layout using one window, to avoid syncronisation problems.
Is there an easy way to generate this alpha key map from my opengl scene?

[This message has been edited by hcatry (edited 11-21-2003).]

[This message has been edited by hcatry (edited 12-08-2003).]

mfort · November 21, 2003, 1:14am

First, render the scene to the texture.
Then draw it once for color channel and once for alpha channel.

BTW: Why split screen?

hcatry · November 21, 2003, 4:55am

Originally posted by mfort:
[b]First, render the scene to the texture.
Then draw it once for color channel and once for alpha channel.

BTW: Why split screen?[/b]

Isn’t one big window faster than 2 seperate windows? (I’m using SDL for windowing)

Do I have to use pbuffers to render a scene as a texture?
Thanks for your input.

ScottManDeath · November 21, 2003, 5:19am

render the scene and after that render a fullscreen white quad with blending enabled and glBlendFunc(GL_DST_ALPHA, GL_ZERO); after that your framebuffer contains the alpha component as a grayscale image. Be sure that you have a pixelformat with alpha

hcatry · November 21, 2003, 5:35am

Originally posted by ScottManDeath:
render the scene and after that render a fullscreen white quad with blending enabled and glBlendFunc(GL_DST_ALPHA, GL_ZERO); after that your framebuffer contains the alpha component as a grayscale image. Be sure that you have a pixelformat with alpha

Interesting technique. But it doesn’t address the problem of needing 2 “windows” per frame: one that contains the current frame rendered normally and a second window that has the grayscale alpha rendering of the frame.
The external video mixer receives the size and position of the 2 windows over a serial link (doesn’t change normally) and uses the first one to overlay the graphics over the video and the second to determine the transparancy of the overlay.

ScottManDeath · November 21, 2003, 5:52am

so the video renderer has direct access to the framebuffer?

serial link is just used to transmit the dimension of the 2 windows. has this video mixer access to for example a pixel buffer? can you control the video mixer using an opengl extension?

hcatry · November 21, 2003, 5:55am

I found this pdf on nVIDIA’s site: “OpenGL Render-to-Texture” http://developer.nvidia.com/attach/3478

While this probably is a solution, it’s complicating my application a lot. Performance is important though.

hcatry · November 21, 2003, 6:00am

[QUOTE]Originally posted by ScottManDeath:
[b]so the video renderer has direct access to the framebuffer?

No, I skipped a step in may explanation. The DVI-I output of the graphics card is send to a Miranda DVI-Ramp (a convertor for DVI to SDI (digital broadcast video norm)

You can see how it works over here: http://www.miranda.com/pdf/datasheet/DVI-Ramp.pdf
on page 2 (broadcast application)

ScottManDeath · November 21, 2003, 6:16am

öhm, now I got it
so you have to have your scene and your alpha key map at the same time om the framebuffer? So render the scene once with color and a smaller size, then do a glCopyPixels to for example the right part of the screen and then do the thing with the quad and blending as mentioned before.

do you have additional control over the dvi ramp?

hcatry · November 21, 2003, 6:45am

Thanks. Using glCopyPixels is a good solution I think. Since it operates within the frame buffer it should be fast, is it?

the dvi-ramp has some other controls but I don’t have the documentation with me, nothing else for controlling the alpha key I think.

do you have additional control over the dvi ramp?[/b]

ScottManDeath · November 21, 2003, 6:53am

maybe you can tell the dvi ramp to take the current frame as rgb and the next as the alpha key, sou in this case you could save the glCopyPixels and just do the blending.

hcatry · November 21, 2003, 6:56am

I’m not sure, but I don’t think there isn’t any alpha information on a video card’s DVI output

Originally posted by ScottManDeath:
maybe you can tell the dvi ramp to take the current frame as rgb and the next as the alpha key, sou in this case you could save the glCopyPixels and just do the blending.

[This message has been edited by hcatry (edited 11-21-2003).]

hcatry · November 25, 2003, 4:44am

I’ve tried this today and it works correctly when I draw only one layer:

black solid background
draw a RGBA texture (logo with some transparancy) using glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA)
Draw the white quad using glBlendFunc(GL_DST_ALPHA, GL_ZERO);

Result: correct grayscale rendering of alpha key

However when I add a solid brackground before the transparant logo, like this:

black solid background
draw a RGBA texture (solid background image) using glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA)
draw a RGBA texture (logo with some transparancy) using glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA)

This gives me the expected blending

Draw the white quad using glBlendFunc(GL_DST_ALPHA, GL_ZERO);

Result: incorrect grayscale rendering of alpha key: still has some gray elements frome the logo while it should be completely white because there is a solid layer (background image) underneath the logo.

Do you know how I can achieve this result?

Originally posted by ScottManDeath:
render the scene and after that render a fullscreen white quad with blending enabled and glBlendFunc(GL_DST_ALPHA, GL_ZERO); after that your framebuffer contains the alpha component as a grayscale image. Be sure that you have a pixelformat with alpha

ScottManDeath · November 25, 2003, 7:23pm

maybe you should clear your framebuffers alpha to a 0.0?

in your second case, which alpha value do you want to keep, the first or the second one?

maybe you have to use the EXT_blend_func_separate extension ( also part of gl 1.4) wich allows separate setting of blending for rgb and alpha

black solid, with alpha 0.0
RGBA tex , glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE,GL_ZERO); --> just uses textures alpha as framebuffers alpha
RGBA tex , glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE,GL_ZERO); --> this keeps the textures alpha in the framebuffer, discarding the old, but still using alpha for the rgb blending
white quad with glBlendFunc(GL_DST_ALPHA, GL_ZERO);

[This message has been edited by ScottManDeath (edited 11-25-2003).]

hcatry · November 26, 2003, 12:13am

Thanks for replying ScottManDeath, I appreciate your expertise

My understanding of using alpha values was that when first rendering (back to front) a pixel with alpha 1 and then a pixel with alpha 0.5 the resulting pixel in the frame buffer would have the blended color values but still have an alpha value of 1. I should not be able to see through anything that I draw with an alpha value of 1. In my case the background is live video.
In case I first draw a pixel with an alpha value of 0.8 and then a pixel with alpha 0.5 I expect the resulting pixel in the framebuffer to have an alpha value of slightly more than 0.8 (compositing 2 transparant objects result in less transparancy).
I have trouble in translating this logic into the grayscale rendering of the alpha key map.

Originally posted by ScottManDeath:
[b]maybe you should clear your framebuffers alpha to a 0.0?

>>> did this

in your second case, which alpha value do you want to keep, the first or the second one?

>>> the first

maybe you have to use the EXT_blend_func_separate extension ( also part of gl 1.4) wich allows separate setting of blending for rgb and alpha

black solid, with alpha 0.0

RGBA tex , glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE,GL_ZERO); --> just uses textures alpha as framebuffers alpha

RGBA tex , glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE,GL_ZERO); --> this keeps the textures alpha in the framebuffer, discarding the old, but still using alpha for the rgb blending

white quad with glBlendFunc(GL_DST_ALPHA, GL_ZERO);

[This message has been edited by ScottManDeath (edited 11-25-2003).][/b]

ScottManDeath · November 26, 2003, 3:53am

you want:

the live video as the background (copied from a texture) [has the video alpha chanel ?]
blend multiple times, also using the alpha chanels of the incoming textures
use ONLY the alpha map from your key texture as a grayscale image without using the alpha of the blendings before as an input

with using the EXT_blend_func_separate extension (or gl 1.4 function calls) you can specify separate blend functions for RGB and ALPHA blending. so you can manipulate the RGB values of the frambuffer while at the same time keep the alpha values of the framebuffer the same.

maybe it helps if you write down your desired blending equatation and then then the behaviour of GL blending

like:
rgb_final = tex0.rgb * tex1.alpha + tex1.rgb
alpha_final = tex0.alpha

gl blending (value = rgb or alpha, using the mentioned extension to make separate settings for rgb/alpha):
fb_value = incoming_value * src_factor + framebuffer_value * dest_factor

where src_factor and dest_factor are like GL_SRC_ALPHA, GL_ONE, GL_DST_ALPHA …

so it gets probably easier to see that you can archieve this by doing follwoing

render tex0
glBlendFuncSeparate( GL_ONE,GL_ZERO,GL_ONE,GL_ZERO);
render tex1
glBlendFuncSeparate(GL_ONE,GL_SRC_ALPHA,GL_ZERO,GL_ONE);

yes I know this looks kinky, but I think after sleeping over it and maybe reading the spec it will become clear …

by the way which video card do you use? if it is a quite decent like GeForce 4 Ti or Radeon 9500 maybe you can use multitextureing or fragment shading to compose your final rgb alpha values without extensive use of blending. this would make some parts easier and would also increase performance a lot, probably

[This message has been edited by ScottManDeath (edited 11-26-2003).]

hcatry · November 26, 2003, 6:53am

I’ve loaded the glBlendFuncSeparateEXT and I’m trying it out right now, but so far the results are “a different kind of blending”. I have to studie more about this stuff for now…

I’m using a Radeon 9700 on my development machiene and a nvidia Quadro FX3000G on the production machiene.

hcatry · December 4, 2003, 2:36am

I finaly got everything working more or less the way I wanted it to.
But now I’m faced with another problem: using nvidia hardware
when I draw the white quad with glBlendFunc(GL_DST_ALPHA, GL_ZERO) to see the alpha buffer in grayscale, I always see a white quad, as if everything drawn before it has alpha=1.
On ATI hardware everything is correct.
Any idea what may cause this?

Originally posted by ScottManDeath:
[b]you want:

the live video as the background (copied from a texture) [has the video alpha chanel ?]

blend multiple times, also using the alpha chanels of the incoming textures

use ONLY the alpha map from your key texture as a grayscale image without using the alpha of the blendings before as an input

with using the EXT_blend_func_separate extension (or gl 1.4 function calls) you can specify separate blend functions for RGB and ALPHA blending. so you can manipulate the RGB values of the frambuffer while at the same time keep the alpha values of the framebuffer the same.

maybe it helps if you write down your desired blending equatation and then then the behaviour of GL blending

like:
rgb_final = tex0.rgb * tex1.alpha + tex1.rgb
alpha_final = tex0.alpha

gl blending (value = rgb or alpha, using the mentioned extension to make separate settings for rgb/alpha):
fb_value = incoming_value * src_factor + framebuffer_value * dest_factor

where src_factor and dest_factor are like GL_SRC_ALPHA, GL_ONE, GL_DST_ALPHA …

so it gets probably easier to see that you can archieve this by doing follwoing

render tex0
glBlendFuncSeparate( GL_ONE,GL_ZERO,GL_ONE,GL_ZERO);

render tex1
glBlendFuncSeparate(GL_ONE,GL_SRC_ALPHA,GL_ZERO,GL_ONE);

yes I know this looks kinky, but I think after sleeping over it and maybe reading the spec it will become clear …

by the way which video card do you use? if it is a quite decent like GeForce 4 Ti or Radeon 9500 maybe you can use multitextureing or fragment shading to compose your final rgb alpha values without extensive use of blending. this would make some parts easier and would also increase performance a lot, probably

[This message has been edited by ScottManDeath (edited 11-26-2003).][/b]

ScottManDeath · December 4, 2003, 5:24am

maybe you should set glClearColor(r,g,b, 0.0);

does this occur wafter you drawed your texture with alpha?

maybe upgrading you driver helps?

testing on an other nv card?

hcatry · December 4, 2003, 5:35am

Originally posted by ScottManDeath:
[b]maybe you should set glClearColor(r,g,b, 0.0);

> did that.

does this occur wafter you drawed your texture with alpha?

yes.

maybe upgrading you driver helps?

testing on an other nv card?[/b]

I’ve tested it on 2 different “nvidia” systems, same problem. ATI was fine
Compositing textures using glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) works fine both on ATI en nVIDIA.