After reading through the Vulkan specs I found no way to copy data between different devices without involving host memory. Am I missing something? Was it deliberately left out or will be provided by a later version or by vendor extensions only?
I took a look at the Mantle docs, and there was explicitly a way of doing peer-to-peer copy, started by opening an already allocated memory on a different device by grOpenPeerMemory, but it has no analogue in Vulkan as far as I see.
Yes, this was in Mantle and it was a conscious decision to omit it from the first version of the Vulkan specification. Given that the vast majority of users (especially mobile devices) have only a single GPU, we felt it would be best not to spend time on this, or to push out a suboptimal solution just to make the schedule. Efficient support of multiple GPUs in a system is something that’s on our agenda and we’ll be considering it carefully once the dust settles on 1.0.
I perfectly understand the weight factors behind such a decision. Coming from HPC backgrounds, we feel a little bit neglected, as we almost exclusively work with multi-device configurations. When it comes to certain image processing solutions or computing with real-time visualizations, graphics + compute workload sharing is a must between such devices and current portable solutions for such problems are not satisfactory.
We’ll be keeping an eye on further Vulkan developments in this area.