Rendering live video performance issue

Hello everybody,

This my first project with OpenGL and I have been reading a lot about it to learn as much as possible, but I think I need a bit of help.

I need to render images coming from a framegrabber card using OpenGL…

The images may have various pixel formats depending on the camera used such as mono8bits, yuv422, mono16bits etc.
To speed up display and lighten the load on the CPU I would like to do all pixel format conversion on the GPu via fragment shaders.

I have made an implementation based On GPU Gems 27,

In a nutshell it’s a filtergraph pattern, where each stage processes an image contained in a source texture and produces a result in a destination texture. The first filter loads the initial image via glteximage2d. The last filter renders the texture to a quad. I use double buffering and swapbuffer().

I have developed the few fragment shaders to do the pixel format conversions required.

I use an nVidia Quaddro graphics card.

It works quite well, however, when looking at the CPU usage I can see that one of the core is nearly maxed out. I have done some investigation and it seems that with vsync enabled a lot of time is spent polling for vsync. I tried disabling vsync and that seems to help but I still occasionally get the CPU usage jump to 100% for indefinite period of time. I don’t understand this as my code is very light and profiling shows most of time is spent n graphics card driver.

Before spending more time Investigating my design I’m wondering whether the filtergraph approach I chose is the right one? Or if this is the wrong way to go, what woukd be a better approach? Does anybody have experience rendering live video?

I have read other posts where people talk about using PBO to upload the image to a texture rather than glteximage2d. I’m not sure to understand the benefit. Can somebody shine some light?