Settings for Hardware GUI BlendMode

Sorry for my lack of vocabulary, I am after all a beginner.

I want to configure my OpenGL application to render at a hardware accelerated level where it will blend with the GUI’s (Windows and MAC) screen using a specific color code as a mask.

I am looking to do a trick similar to what you can do with VLC media player and Media Player Classic where you can start media, minimize the player, then color the background of a terminal window (0x000001 for MPC, 0x000010 for VLC) and anything left that color after the GUI has rendered will be replaced at the hardware level with the bits from the media.

Is this possible to do with hardware accelerated OpenGL? I want to be able to display the OpenGL behind other applications, of which I do not have access to the source or any plugin SDK, but I do have access to the background color.

I used to do such a thing in the late 90’s with Video games using a “render as desktop” mode.

Any help with the OpenGL settings / configuration would be greatly appreciated,

I’ve been doing more research on what I want to do, and the vocabulary I’m looking for is a “hardware overlay”.

It doesn’t look like OpenGL will support this? Is this correct? But a combination of OpenGL and Quark may? I saw where QT OpenGL supports hardware overlays…

AFAIK there is no way using consumer hardware to do GL hardware overlay planes.

This may work, but very convoluted :

  • draw GL rendering on FBO
  • retrieve rendering with glReadPixels
  • write that to a “normal” 2D hardware overlay.

Thank you for the response. I appreciate your taking the time to help me with my question.

So I figured maybe I could:

a) Read in the OpenGL from a specific program.
b) Write that into my own buffer.
c) Draw my own stuff over the buffer (but only when a specific color is in the first buffer)

For A) I know there are programs that are capable of capturing the output of a specific program. I just need to figure out the calls.

C) Perhaps a specific shader? Or is there a way to join the pixels of two buffers and/or texture2ds with a custom pixel selection algorithm?

for each pixel being merged:

If pixelInOriginal's color is <specific-pass-through-color>
    pixelOut = pixelInCustom
    pixelOut = pixelInOriginal;