Sorry for not posting the reply here as well (I asked on flipcode also), but I came up with a method which gave me exactly what I was looking for. Here is the post.

After trying many methods, I finally found a method which works perfectly. Given your lat/lon position, heading(yaw), and pitch/roll, you can calculate an equivalent spherical rotation as follows.

SphericalPosition is a function that returns the lat/lon position with height projected onto a sphere of a given radius.

I tried using the forum code stuff, but it messed up my formating (got rid of spaces), so here it is as is.

/* Get the origin */

Vector3 ORIGIN = SphericalPosition( lon, lat, 0 );

/* Generate the up vector */

Vector3 UP = SphericalPosition( lon, lat, 100 );

/* Generate the forward vector */

double radius = 0.01 (keep relatively small since we are dealing with lat/lon coordinates)

double cs = cos(heading)*radius;

double sn = sin(heading)*radius;

Vector3 FORWARD = SphericalPosition( lon+cs, lat+sn, 0 );

/* Generate the right vector */*

heading += 90RADIAN; /* Rotate the heading by 90 degrees to get the right vector */

radius = 0.01 (keep relatively small since we are dealing with lat/lon coordinates)

cs = cos(heading)*radius;

sn = sin(heading)*radius;

Vector3 RIGHT = SphericalPosition( lon+cs, lat+sn, 0 );

/*

** Pitch is applied by rotating the up and forward vector around the right vector.

*/

UP = RotateAroundVector( pitch, RIGHT, UP );

FORWARD = RotateAroundVector( pitch, RIGHT, FORWARD );

/* Roll will be something similar, but a rotation around the forward axis */

/*

** Now you have your right/forward/up vector. Use that to compute

** a transformation matrix, and now you have seamless

** cartesian/spherical roaming. Pretty sweet.

*/