I need to transform an object using orientation vectors and a position point (translation). Would it be possible to use a new frame of reference and a position point both relative to its original local frame of reference and origin?

What exactly I want to do is being able to transform the object in 3D without having to figure out angles and rotations, just feed an orientation vectors/matrix.

Yes, the transform from frame B to frame A is matrix M where the column vectors of M are basis vectors of frame B expressed in coordinates of system A with 4th column being the origin of B with respect to A. So if you assume A is your standard cartesian frame of reference and B is one that’s rotated by 45 degrees about z and at some point P. Basis vectors of B are:
[1,1,0]
[-1,1,0]
[0,0,1]
P
Then the transform is
|1 -1 0 Px|
|1 1 0 Py|
|0 0 1 Pz|
|0 0 0 1|

Edit:
oops, I think I got the basis vectors wrong. They are not normalized!

Thanks. Got it! so i need to find the orthogonal basis of the new vectors relative to the original basis, then apply the inverse to this matrix, which is indeed the transpose.

Depends on what system you’re transforming from. If your objects are in frame B then to express them in frame A you simply multiply by the matrix. Which will be equivalent to rotating an object in cartesian frame A by 45 degrees and translating for example.

The inverse is not the transpose because you have 4 columns - the the origin. This makes the matrix non orthogonal. Only in certain cases when it’s 0,0,0,1 you can kind of “ignore” the 4th column and take the transpose to get the “inverse”

Can you clarify how you’d make
1 0 0 1
0 1 0 1
0 0 1 1
0 0 0 1
Orthogonal to invert by transpose? This is a frame with origin 1,1,1 with respect to Cartesian frame…