It sounds like what you are trying to find is the eye in Local Coordinates of your original object, not in global fixed coordinates. Is that correct?
Take a look at OpenGL Transformations and scroll down to section “Eye Coordinates”. When you scale and rotate your object you are computing the ModelView MAtrix. You can get that matrix with code
then since [x,y,z,w]_eye = (0,0,0,1), you can compute your eye location in local object coordinates with your [x,y,z] = Inverse(m)[x,y,z,w]_eye=Inverse(m)[0,0,0,1]. This is messy because you have to compute the inverse of m manually.
This is a common problem, so common that OpenGL provides a better way to think of the eye/camera and locating objects in the scene – gluLookAt. gluLookAt places your camera where you want in relation to your fixed scaled object. You can then think about moving the camera that way you always “know” the camera position explicitly. Try looking at OpenGL Redbook Chapter 3. and give it a good read. It is well worth the effort.
There was another post that may be helpful explaining gluLookAt after you read Chapter 3 – see Post263134