Real-world camera parameters

OpenGL OpenGL: Advanced Coding
1

Hi Guys,

I have Matlab code that renders (albiet very slowly) 3d points from the POV of a camera that is based on a real camera. The data originated from a stereo capture, and preserving the original camera’s parameters is important. The camera model folows the opencv camera model:


http://opencv.willowgarage.com/documentation/camera_calibration_and_3d_reconstruction.html

The only difference is that I dont multiply by s, and do a divide by z afterwards, which works just as well.

Now, I have all the parameters, cx, cy, fx, fy, and the Rotation/Translation matrix. However, Im not sure how to use these parameters in OpenGL. I’ve tried fiddling around with glFrustum and gluPerspective. I’ve even tried loading the perspective matrix (the one with fx and cx parameters) via glMultMatrixd and glLoadMatrixd but i keep getting a black screen.

In short, how do I convert this camera model into something that opengl expects?

BTW It works fine in Matlab, so the camera model is fine.

Thanks in advance.
Fugi

2

I think that the simplest solution would be to write a vertex shader with glsl and perform your custom camera transformations.

If you want to use the fixed pipeline, you will have to rewrite your matrices before since opengl uses only 4x4 matrices.

[R|t] is the camera view transformation, consequently you can set the opengl modelview matrix with this one. Assuming transformed point are already in world space, it would look like this:


ModelViewMatrix = [ r11 r12 r13 t1
                    r21 r22 r23 t2 
                    r31 r32 r33 t3
                     0   0   0   1 ]

Then for the projection transformation set the opengl projection matrix with something like this:


ProjectionMatrix = [ fx  0  cx 0
                      0 fy  cy 0
                      0  0  1  0
                      0  0 -1  0 ]

After projection transformation, opengl automatically performs perspective division: (x,y,z,w) -> (x/w, y/w, z/w, 1)

glFrustum or gluPerspective won’t help here to build this kind of transformation so build your own matrices and load them with glLoadMatrix.

Note that after perspective division with the projection matrix above, the depth information is lost and z is always equal to 1.

The glFrustum function builds a matrix that compute a non linear transformation to keep this depth information but it has its drawbacks. See:
http://www.opengl.org/sdk/docs/man/xhtml/glFrustum.xml

3

Thanks dletozeun,

Actually, as I mentioned earlier, I already tried glLoadMatrix, precisely the way you mentioned. The model is already transformed correctly, and can be viewed fine with an orthographic camera, yet, the projection matrix produces a blank screen.

I think I have found the answer, however. For those of you facing a similar problem, it is explained in detail here:

http://old.uvr.gist.ac.kr/wlee/web/techReports/ar/Camera%20Models.html

For the impatient, here is the summary:



where u0 and v0 are cx and cy.

I have not tried it yet, though I will report back if it works.

4

I would like to report that after implementing the above technique myself I can safely say that it does work.

If you are getting a black screen, then first try setting your viewport to 0,0,screenwidth,screenheight (to make sure nothing is clipped out), and then rotating/transforming your model until you see something. This is likely because (as in my case) the camera follows a true pinhole camera model and the projection actually inverts the image, and the camera faces the oppossite direction (i.e, the image is formed behind the lense)…