OpenGL + OpenCV (Computer Vision)

Ok, so as the title suggests, I’m trying to combine OpenGL with OpenCV (Computer Vision).

Originally, I was just trying to track objects based on color using OpenCV. I actually have that working pretty well. I have a Playstation Move controller hooked up via bluetooth, I can set the color, and track it.

After I got that working, I thought it’d be cool to integrate my OpenCV project with a simple OpenGL program. The idea was to have my object tracking code update a sphere created in OpenGL and move it on screen in relation to the user’s movements.

So here’s what I’ve got: (Disclaimer: I’m very new to OpenGL)

Inside my openGL display function:
[ul][li]I find and track the circle created by the Playstation Move controller on screen. (I do this by finding the center of the circle) []On each frame I compare the x and y values of the current center of the circle to the x and y values of the center found in the previous frame. This is how I determine movement / adjust the openGL program.[]If the center has changed by at least 4 pixels in any direction I consider that movement and adjust the sphere on screen accordingly.[/ul][/li]So, that’s the basic idea. Determine movement by comparing x and y values and translate the sphere in the appropriate direction.

This seems to work “ok”, at best. I get reasonable results if I move the controller slowly and only in left/right or up/down directions. It does respond to diagonal movement, but it doesn’t look nearly as smooth. I get horrible results if I try to move the controller at any kind of fast pace. The sphere on screen just sort of freaks out and jerks around trying to keep up, but definitely does not respond in a natural way.

I’m posting this here because my OpenCV/object tracking code seems to work, my issue seems to lie in finding a way to get the results from openCV to OpenGL in a format that is meaningful.

Am I taking this in the right direction? I can’t help but think that some of my issues come from trying to relate differences in pixel locations to translations in OpenGL. I assume there’s no way that I can relate my openCV coordinate system (0,0 in top left corner) to openGL and just draw the sphere at the correct coordinates automatically, right? If not, and translating is the only way to go, how should I be approaching this?

I hope this makes sense. I tried to break down the problem as much as I could without writing a book, heh.

Any help is appreciated.

If you just want to use pixel coordinates to draw 2D with OpenGL, it is in fact very easy:

glMatrixMode (GL_PROJECTION);
glLoadIdentity ();
glOrtho (0, windowWidth, windowHeight, 0, -1, 1);
glMatrixMode (GL_MODELVIEW);
Then draw in pixels, from top left corner.

My gut feeling is that your camera is crappy/too slow/not sensitive enough and you end up trying to track a smudged blur instead of a bright colored circle whenever the movement is too fast.
Did you disable the automatic brightness ? Try to lower exposure (in practice this often reduces the shutter time), push fps as high as you can, lower resolution, all these can help.
What are you camera specs ? The PS Eye is said to be able to do 60Hz 640x480, 120Hz 320x240, and that helps tracking a lot.

Yea, it is entirely possible my camera is causing some of my issues. It only runs at 30Hz. I may look into picking up a PS eye to see if that corrects some of my problems.

I was operating on the assumption that my camera was decent enough since my OpenCV program does track the glowing sphere pretty well, but the more I think about it the more I see where it could be sending some bogus information to my OpenGL program when I’m moving the sphere at a fast pace. (eg: when I move the sphere at a fast pace it turns into a long white streak on screen as opposed to the circle. I can see where that would return an incorrect center of a “circle”, hehe.)

Thank you for the tip about drawing with pixel coordinates in 2D. Hopefully I can integrate that and eliminate some of the differences I’m getting in trying to convert movement in OpenCV to an appropriate translation value in OpenGL.

Thank you for your time.