Memory error with glsl shaders

#1

Hi, for the pass few weeks I decided to try Vulkan compute shader to acceleration my app.
I am following the instruction in book Vulkan Programming Guide by Graham Sellers.

every thing seem to go right until the moment I try to create a pipeline that does more than the test module.

main ()
{
// do nothing.
}

but when I try to compile a more meaningful shader, I get error VK_ERROR_OUT_OF_HOST_MEMORY on the call
err = vkCreateComputePipelines(context.m_device, context.m_pipeLineCache, 1, &computePipeLineInfo, &context.m_allocators, &context.m_initBodyPipeLine);
dgAssert(err == VK_SUCCESS);

my application is open source and can be check out in git-hub


if anyone want to verify and see what I am doing wrong.
it has a visual studio solutions and a cmake script that build out of the box.
it is also set to to reproduce the error immediately.
my system is an Intel core i7 7700 with an amd rx 480 driver 19.1.1

at this point I have not idea what to do, because it seems the error is a function of how many instructions are in the shader. at time it even seem random.

the vulkan SDK is not providing me with any feedback information as to what is wrong.

Thank you.

#2

Hi Julio,

it looks like the problem is caused by how you load the shaders in dgWorldBase::CreateShaderModule. Stepping through that code I noticed that the shaderByteCode only contained zeroes, which on my NVidia GPU already failed at vkCreateShaderModule.

Quickly replacing this with some of my own SPIR-V shader loading code like this:

	std::ifstream is(std::string(m_libPath) + std::string(shaderName) + ".spv", std::ios::binary | std::ios::in | std::ios::ate);

	if (!is.is_open()) {
		return VK_NULL_HANDLE;
	}

	size_t size = is.tellg();
	is.seekg(0, std::ios::beg);
	char* shaderCode = new char[size];
	is.read(shaderCode, size);
	is.close();

	VkShaderModuleCreateInfo shaderModuleCreateInfo;
	Clear(&shaderModuleCreateInfo);
	shaderModuleCreateInfo.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO;
	shaderModuleCreateInfo.codeSize = size;
	shaderModuleCreateInfo.pCode = (uint32_t*)shaderCode;

Will get the shader module to compile as expected, and the vkCreateComputePipelines then also compiles fine with me, even with the full shader code.

I could not test further though, as I get a crash later on in some other Newton related function, but as the problem seems to be how the shader is loaded this fix will help you continue with implementing Vulkan.

P.S. : Nice to see you doing some Vulkan related stuff :slight_smile:

#3

oh thank for the replay, Yes me and my friend Dave figured out what was wrong, i was setting the
binding to zero in VkDescriptorSetLayoutCreateInfo which apparently works with a dommy shader but fail the moment you add a binding to a shader.
The strange thing is that it seems to worked on nvidia but not with AMD drivers.
Anyway I was doing it incorrectly, but now is fine.

on the vulkan stuff yes, I have waited for almost 10 years for an api that does not have some much baggage. I hope vulkan do not go the way opencl did where every vendor made their own SDK incompatible with other vendor SDKs.
I really like the granularity of Vulkan, maybe too much but that’s just a learning curve, I imagine.
anyway I will continue doing this, see how far I get.

thank you for the interest. I will post later if I get stuck or when I get some results.