Ok, this might get complicated but I’ll do my best to explain as good as I can:

I have an OpenGL (JOGL) scene consisting of a white plane with 4 colored orbs on it (red, green, blue, yellow). My assignment is simple: given the world coordinates of the orbs, calculate the world position of the camera from which the orbs are viewed at time t.

For this I am using the Levenberg-Marquardt algorithm (no need to look it up, not relevant to this question), which basically is a trial and error method that requires me to identify the camera coordinates of the (centers of the) orbs for every iteration in order to calculate the result with the smallest offset, given a set of values.

I have already implemented the LM algorithm, no problems there. However, this algorithm only gives me a possible solution, it is my job to find out if this solution is correct or not. In my case, this means to transform 4 points (the centers of the orbs) from world coordinate system into image coordinate system, using a set of parameters given by the algorithm. And this last part is what’s troubling me for a week or so now.

It seems that, no matter what I do, I can’t get this stuff right. The world coordinates of the points are known and the camera translation and orientation are given by the algorithm. The problem is that I don’t have the correct intrinsic matrix to do this transformation. My results should be something like this:

1 = 175.0, 267.0

2 = 198.0, 311.0

3 = 291.0, 266.0

4 = 385.0, 289.0

so basically 4 (x,y) coordinate pairs in a 580x560 window.

Here are some of the things I tried:

If I try to identify the intrinsic matrix that OpenGL uses to render my view using this code

```
float stack[] = new float[16];
gl.glGetFloatv(gl.GL_PROJECTION_MATRIX, stack, 0);
```

I get the following result, which is about as wrong as it can possibly get:

```
{2, 0, 0, 0}
{0, 2, 0, 0}
{0, 0, 1, 0}
```

results =

x = 0.8, y = 0.64

x = 1.52, y = 0.56

x = 1.2, y = 0.64

x = 0.88, y = 0.48

I also tried loading my own intrinsic matrix using this code

```
float[] intrinsic = makeProjectionMatrix(); // fill with predetermined values
gl.glMatrixMode(gl.GL_PROJECTION);
gl.glLoadIdentity();
gl.glLoadMatrixf( projectionMatrix, 0 );
gl.glMatrixMode(gl.GL_MODELVIEW);
gl.glLoadIdentity();
gl.glScalef(2/width, 2/height, 1);
```

but this screws up my view so bad I can’t even see my plane anymore or it removes all but one of my orbs from the plane, making this solution completely useless… This could mean that I’m not using the correct values, but I’ll honestly admit that I don’t know enough about this stuff to actually know what I should or shouldn’t use here. I’ve read extensively on the subject, though I haven’t been making a lot of progress, unfortunately.

Any help on this subject would be much appreciated, I don’t have a clue what to do.