Using Tensor cores and RT cores

[user1437]

Hi,

I am interested in doing ray tracing along with efficient matrix multiplication in Vulkan using an NVIDIA GPU.

• Raytracing : https://nvpro-samples.github.io/vk_raytracing_tutorial_KHR/
• Matrix multiplication : https://developer.nvidia.com/blog/machine-learning-acceleration-vulkan-cooperative-matrices/

This is not supported in Optix by design. [url=https://forums.developer.nvidia.com/t/using-tensor-cores-in-optix/202239]https://forums.developer.nvidia.com/t/using-tensor-cores-in-optix/202239[/url]

The closest hit shader pseudo code would look something like this

int main() {
  
  // .....

  // trace shadow rays
  traceRay(); // leverage RT cores for this

  loadMatrixDataIntoSharedMemory();
  synchronize();

  // compute BSDF if surface is not occuled
  peformMatrixMultiplication(); // leverage Tensor cores for this  
}

I am mostly interested in this megakernel approach although there might be other ways to do this like the wavefront approach. [url=https://forums.developer.nvidia.com/t/deep-learning-and-optix/112722/2]https://forums.developer.nvidia.com/t/deep-learning-and-optix/112722/2[/url].

I am very new to Vulkan and wanted to know if something like this is even possible? Considering the learning curve, I thought it would be better to get a high level idea before jumping into learning Vulkan and implementing it. I understand that this question is highly unusual, please let know if more information is needed.

I was not sure if this question belongs here or on the NVIDIA forums. Apologies if it doesn’t belong here.

Thanks

[Dark_Photon]

@user1437

I was not sure if this question belongs here or on the NVIDIA forums. Apologies if it doesn’t belong here.

Either is fine. These forums are for using Vulkan in general, with or without vendor extensions. But if you don’t get a response here, definitely try the NVIDIA forums. NVIDIA support folks will sometimes chime in there.