Generating vertex array in Vulkan to be rendered by OpenGL

I apologize if this should have been in Vulkan rather than OpenGL, as my problem involves both.

I have an existing library I created using Vulkan compute shaders to generate terrain meshes. I’m trying to write my own render in using OpenGlL to render these meshes by directly copying the mesh data from the Vulkan compute buffers with the vertex data into my Vertex Buffer Object bound to my Vertex Array Object.

I have a vulkan kernel the stitches the vertex,normal,color,texcoord arrays into the format specified by my vertex attributes, which I’ve confirmed works fine and I can extract the data back to host memory in the correct form. I could, technically, just set the VBO with this host buffer with glBufferData, but I would much rather use “EXT_external_objects”/“VK_KHR_external_memory” extensions to keep it on the GPU (as this is the entire reason I’m writing the rendering from scratch).

I’ve set the VkBuffer to allow external memory:

void Utilities::CreateBuffer(
    VkPhysicalDevice& physicalDevice, 
    VkDevice& device, 
    VkDeviceSize size, 
    VkBufferUsageFlags usage, 
    VkSharingMode sharingMode,
    bool external,
    VkBufferCreateFlags flags, 
    VkMemoryPropertyFlags properties, 
    std::vector<uint32_t>& queueFamilies, 
    VkBuffer& buffer, 
    VkDeviceMemory& bufferMemory)
{
    VkBufferCreateInfo bufferInfo = Utilities::getBufferCreateInfo(
        size, usage, sharingMode, flags, queueFamilies);

#ifdef WIN32
    VkExternalMemoryHandleTypeFlags type = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT_KHR;
#else
    VkExternalMemoryHandleTypeFlags type = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT_KHR;
#endif

    VkExternalMemoryBufferCreateInfo externalInfo = {
        .sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_BUFFER_CREATE_INFO,
        .pNext = NULL,
        .handleTypes = type
    };

    if (external)
        bufferInfo.pNext = &externalInfo;


    if (vkCreateBuffer(device, &bufferInfo, nullptr, &buffer) != VK_SUCCESS) {
        throw std::runtime_error("Failed to create buffer!");
    }

    VkMemoryRequirements memRequirements;
    vkGetBufferMemoryRequirements(device, buffer, &memRequirements);

    //VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT
    //VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT

    VkExportMemoryAllocateInfo exportInfo = {
        .sType = VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO,
        .handleTypes = type
    };

    VkMemoryAllocateInfo allocInfo{};
    allocInfo.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO;
    allocInfo.allocationSize = memRequirements.size;
    allocInfo.memoryTypeIndex = findMemoryType(physicalDevice, memRequirements.memoryTypeBits, properties);

    if (external)
        allocInfo.pNext = &exportInfo;

    if (vkAllocateMemory(device, &allocInfo, nullptr, &bufferMemory) != VK_SUCCESS) {
        throw std::runtime_error("Failed to allocate buffer memory!");
    }

    vkBindBufferMemory(device, buffer, bufferMemory, 0);
}



......


	switch (mType) {
	case Buffer_Type::READ:
		mStage_transfer_flag = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
		mTransfer_flag = VK_BUFFER_USAGE_TRANSFER_DST_BIT;
		break;

	case Buffer_Type::Write:
		mTransfer_flag = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
		mStage_transfer_flag = VK_BUFFER_USAGE_TRANSFER_DST_BIT;
		break;

	case Buffer_Type::Read_Write:
		mTransfer_flag = VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
		mStage_transfer_flag = VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
		break;
	}



	Utilities::CreateBuffer(
		*mPhysicalDevice,
		*mLogicalDevice,
		mSize,
		VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_VERTEX_BUFFER_BIT | mTransfer_flag,
		VK_SHARING_MODE_CONCURRENT, external,
		0,
		VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT,// VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT, //VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT
		mAllQueueFamilies,
		mBuffer,
		mBufferMemory
	);

I’ve created the memory object and imported the memory handle into it:

void ComputeBuffer::initExternalCopy()
{
	glCreateMemoryObjectsEXT(1, &mExternalMemObj);

	VkMemoryRequirements req{};
	vkGetBufferMemoryRequirements(*mLogicalDevice, mBuffer, &req);
	
#ifdef WIN32

	PFN_vkGetMemoryWin32HandleKHR vkGetMemoryWin32HandleKHR = mContext->get_GetMemoryWin32_func();

	VkMemoryGetWin32HandleInfoKHR handleInfo = {
		.sType = VK_STRUCTURE_TYPE_MEMORY_GET_WIN32_HANDLE_INFO_KHR,
		.pNext = NULL,
		.memory = mBufferMemory,
		.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT_KHR
	};

	vkGetMemoryWin32HandleKHR(*mLogicalDevice, &handleInfo, &mFD);

	glImportMemoryWin32HandleEXT(mExternalMemObj, mSize, GL_HANDLE_TYPE_OPAQUE_WIN32_EXT, mFD);

#else
	VkMemoryGetFdInfoKHR fdInfo = {
		.sType = VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR,
		.memory = mBufferMemory,
		.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT
	};

	vkGetMemoryFdKHR(*mLogicalDevice, &fdInfo, &mFD);

	glImportMemoryFdEXT(mExternalMemObj, mSize, GL_HANDLE_TYPE_OPAQUE_FD_EXT, mFD);
#endif

	glGenBuffers(1, &mExternalBuffer);
	glNamedBufferStorageMemEXT(mExternalBuffer, req.size, mExternalMemObj, 0);

}

And Finally I’ve attached it to the VAO:

void Mesh::Load(IComputeBuffer* buffer)
{
	size_t size = buffer->GetSize();
	//Logger::LogDebug(LOG_POS("Load"), "Load buffer size: %i", (int)size);


	glBindVertexArray(VAO);

	glBindBuffer(GL_ARRAY_BUFFER, VBO);
	glBufferStorageMemEXT(GL_ARRAY_BUFFER, size, buffer->External_Memory(), 0);

	
	// position attribute
	glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 11 * sizeof(float), (void*)0);
	glEnableVertexAttribArray(0);
	// normal attribute
	glVertexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, 11 * sizeof(float), (void*)(3 * sizeof(float)));
	glEnableVertexAttribArray(1);
	// color attribute
	glVertexAttribPointer(2, 3, GL_FLOAT, GL_FALSE, 11 * sizeof(float), (void*)(6 * sizeof(float)));
	glEnableVertexAttribArray(2);
	// texture coord attribute
	glVertexAttribPointer(3, 2, GL_FLOAT, GL_FALSE, 11 * sizeof(float), (void*)(9 * sizeof(float)));
	glEnableVertexAttribArray(3);


	glBindVertexArray(0);
	glBindBuffer(GL_ARRAY_BUFFER, 0);
}

However, the object still does not render to the screen. I’m not sure what else I would need to do to enable sharing the vertex buffer.

EDIT: Investigating further, “glImportMemoryWin32HandleEXT” triggers a “GL_OUT_OF_MEMORY” error.

Any help is appreciated!

Thanks

Have you tried running Nvidia’s example, does that one work for you?

Yea, just tried running the sample and it works fine. I had seen that before, but will dig deeper into it to see if I can find where mine is off.

The example project runs. However, I statically linked to nvpro_core as a library and attempted to create the buffer exactly how they do pretty much verbatim, and I’m still getting the GL_OUT_OF_MEMORY error on the ImportMemoryWin32HandleEXT call.

I’m afraid I don’t have much else to suggest. Maybe use a tool (e.g. renderdoc) to capture all OpenGL/Vulkan calls the example and your application make to compare the exact sequence of calls?

I found when I reduce the nvpro example down to a single function and run it completely contained inside the nvpro_core library from my project, it works fine. However, if I pass my own VkInstance/VkDevice/VkPhysical device into that function, I get additional errors relating to the physical device, so I think I may not be setting the device up the same way. I’m still investigating this to to see how mine differs. If I can’t get my instance/devices to work with it, I’ll probably just refactor my vulkan class to rely completely on nvpro_core.