Help sharing VRAM between Vulkan instances in 2 separate processes

Hi! If someone could provide some insight here I would be incredibly grateful! (I’ve been banging my head with this problem for a few days now, even though it does not seem overly complicated…)

TLDR, I am trying to share a GPU memory object between two (non-related) processes on Linux. Each process manages a single Vulkan instance. Shared memory is created on a WRITER app, by passing VkExternalMemoryBufferCreateInfo as a struct extension to VkBufferCreateInfo. I get the file descriptor (FD) that references the shared memory by calling vkGetMemoryFdKHR. So far so good.

I then pass the FD to the READER app by means of a Unix Domain socket (I am running Ubuntu). The FD seems to be received correctly by the READER, which then passes it to VkImportMemoryFdInfoKHR, which in turn gets added to VkMemoryAllocateInfo when creating the buffer on the client.

After both WRITER and READER are up and running, both having instantiated their buffers using the duplicated FD, I make WRITER write (duh) a stream of bytes to the buffer. I do this by calling vkMapMemory, writing some data to the mapped buffer, then calling vkUnmapMemory. I get no errors whatsoever (I have validation layers setup properly on both sides). After this WRITER pauses.

I then make READER try to read the stream of bytes. vkMapMemory → read data from buffer → vkUnmapMemory. Again I get no errors. HOWEVER, on the client, no matter what I try, I always get a stream of zeroes. There is no glimpse of the data that was written on the WRITER side. :frowning:

I have tried playing around with some extra features (like invoking vkFlushMappedMemoryRanges after writing to memory), but to no avail. My intuition is I am making some silly mistake - given I am an absolute Vulkan noob that is to be expected. Even so, none of my error checking seems to be catching it.

I have posted up a most minimal example of what’s described above in this repo:

  • writer.cpp contains server-specific code
  • reader.cpp contains client-specific code
  • common.cpp contains resources shared by client and server modules

If a good soul could help me crack this I would buy them a coffee and flow them with eternal gratitude!

I figured out my code above works fine on a different computer with more recent Nvidia drivers. This issue comes up only on my working laptop, which has a downgraded kernel version (5.4) and pinned down Nvidia drivers + CUDA versions.