let’s say I’ve got an object bounding box and a ray. And I want to test wether the ray
intersects the box or not. So I take the inverse matrix of the obb and multiply the
ray’s position and ray’s direction with it. Then I perform a ray aabb intersection. The obb
has been modeled in object space and so I can use this as an aabb for the intersection test.
So far that works. But I remember that normals should rotated by the inverse transpose
of the model matrix. And I think we can interpret the ray direction as a normal. Since the
ray position is a position I tried to multiply the ray position with the inverse and the ray direction
with the inverse transpose. But then there is no intersection. If I multiply both, the ray position
and the ray direction with the inverse or the inverse transpose then there is an intersection.
For the case of ray obb intersection in the way described above, do I need the inverse transpose ?
It only works if the ray position and ray direction gets multiplied with the same matrix (inverse or inverse transpose).
But it doesn’t work if the ray position us multiplied with the inverse and the ray direction is multiplied with the inverse transpose.
and thanks for the fast answer. But I have one more question according to your point, that the ray direction is a position vector.
The vector that points to the ray’s origin is a position vector, so far I agree. But how can the direction vector be a position vector as well ?
Or did I miss some facts ?
If I’m wrong please let me know. Thanks again for your help.
It’s the position of a point on the ray relative to another point on the ray. But I disagree with its characterisation as a “position vector”.
The main issue is that it isn’t a normal, and shouldn’t be transformed using the normal matrix (inverse-transpose).
However, it is a direction rather than an absolute position, so if it’s a homogeneous (4D) vector, its w component should be zero. This ensures that transforming it by a 4x4 matrix won’t apply any transformation component.
The reason for using the inverse-transpose matrix for normals is that the normal vector will remain to the surface even if the original matrix isn’t orthonormal. But this only applies to normals. And if the original matrix is orthonormal, then the inverse is just the transpose, so the inverse-transpose matrix is identical to the original matrix.
Finally, if your transformation includes perspective, you shouldn’t try to represent the ray as a position and direction; use two positions instead. IOW, you need to transform two points on the ray, divide by w, then (if necessary) subtract the two to obtain the direction. Subtracting then transforming the resulting direction vector won’t work.