Adreno 506 vkCreateComputePipelines has returned VK_INCOMPLETE

#1

Hi All, I have problem on Adreno 506.

vkCreateComputePipelines has returned VK_INCOMPLETE. I have no error message from Vulkan debug layers,
but i see an error message from Adreno Driver(Android Studio/logcat):

31569-31637/com.qualcomm.vulkan.compute.d I/Adreno: Shader compilation failed for shaderType: 5

Sample code:

    void VkSample::Test(VkDevice mDevice)
    {
    	const VkDescriptorSetLayoutBinding vkDescriptorSetLayoutBinding[4] = {
    			{0, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT, NULL },
    			{1, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT, NULL },
    			{2, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT, NULL },
    			{3, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT, NULL },
    	};

    	VkDescriptorSetLayoutCreateInfo vkDescriptorSetLayoutCreateInfo = {};
    	vkDescriptorSetLayoutCreateInfo.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO;
    	vkDescriptorSetLayoutCreateInfo.bindingCount = 4;
    	vkDescriptorSetLayoutCreateInfo.pBindings = vkDescriptorSetLayoutBinding;

    	VkDescriptorSetLayout vkDescriptorSetLayout = VK_NULL_HANDLE;
    	VkResult res =  vkCreateDescriptorSetLayout(mDevice, &vkDescriptorSetLayoutCreateInfo, NULL, &vkDescriptorSetLayout);

    	VkPipelineLayout vkPipelineLayout = VK_NULL_HANDLE;

    	VkPipelineLayoutCreateInfo vkPipelineLayoutCreateInfo = {};

    	vkPipelineLayoutCreateInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO;
    	vkPipelineLayoutCreateInfo.setLayoutCount = 1;
    	vkPipelineLayoutCreateInfo.pSetLayouts = &vkDescriptorSetLayout;

    	res =  vkCreatePipelineLayout(mDevice, &vkPipelineLayoutCreateInfo, NULL, &vkPipelineLayout);

    	VkComputePipelineCreateInfo vkComputePipelineCreateInfo = {};

    	VkPipelineShaderStageCreateInfo& vkPipelineShaderStageCreateInfo = vkComputePipelineCreateInfo.stage;
    	vkPipelineShaderStageCreateInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO;
    	vkPipelineShaderStageCreateInfo.flags = 0;
    	vkPipelineShaderStageCreateInfo.pNext = NULL;
    	vkPipelineShaderStageCreateInfo.stage = VK_SHADER_STAGE_COMPUTE_BIT;
    	vkPipelineShaderStageCreateInfo.module = CreateShaderModuleFromAsset("shaders/spirv.spv");
    	vkPipelineShaderStageCreateInfo.pName = "CullObjects";
    	vkPipelineShaderStageCreateInfo.pSpecializationInfo = NULL;

    	vkComputePipelineCreateInfo.sType = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO;
    	vkComputePipelineCreateInfo.layout = vkPipelineLayout;
    	VkPipeline vkComputePipeline = VK_NULL_HANDLE;

    	res =  vkCreateComputePipelines(mDevice, NULL, 1, &vkComputePipelineCreateInfo, NULL, &vkComputePipeline);
    }

Original Shader was compiled using glslang tool(https://github.com/KhronosGroup/glslang)

Simple project(Android Studio) are based on Vulkan-Compute sample from Adreno Vulkan SDK:

See Sample.cpp, line 634, method void VkSample::Test. I attached original shader CSShaderGPUInstanceCulling.hlsl, Spir-v shader:
spirv.spv

I have no problem on Mali G71, G76, AMD(Windows/Linux), nVidia(Windows), Intel(Windows)

What do you think about it ?

#2

From my experience with lower-spec Adreno SOCs this may be a driver bug, so unless the device can be updated with newer drivers you may be out of luck.

But can you check if you see something like “location/component mismatch” in adb after the shader compilation failure?

And are you using spirv-opt to optimize SPIR-V? If so, try without this.

#3

Hi @Sascha_Willems, how should me check something like “location/component mismatch” using adb ?

No, i use code are based on GlslangToSpv function from glslang tool:

	glslang::TShader shader(EShLangCompute);
    	glslang::TProgram program;

        // Enable SPIR-V and Vulkan rules when parsing GLSL/HLSL
        EShMessages messages = static_cast<EShMessages>(EShMsgSpvRules | EShMsgVulkanRules | EShMsgReadHlsl);
        shader.setEntryPoint(entryPoint);
        shader.setStrings(shaderStrings, n);
        shader.setHlslIoMapping(true);

        result.size = 0;
        result.data = NULL;
        if (!shader.parse(&Resources, 100, false, messages)) {
        	Helper::OutInfo(shader, "parse");
        	return false;  // something didn't work
        }

        program.addShader(&shader);

        //
        // Program-level processing...
        //

        if (!program.link(messages)) {
        	Helper::OutInfo(shader, "link");
        	return false;
        }
        static std::vector<unsigned int> spirv;
        spirv.clear();
    	glslang::GlslangToSpv(*program.getIntermediate(stage), spirv);
#4

I fixed compute shader issue for Adreno 506. I should use the same compilation options that is used in glslangvalidator.exe. I should fix code used for shader compilation with glslang tool. Anyway Adreno has a bug inside driver.

#5

I’m seeing a similar issue with some of our OpenCL shader modules compiled with the latest revision of clspv (https://github.com/google/clspv).

The Adreno GPU’s return VK_INCOMPLETE, while some other phones we have with ARM GPU’s (Samsung Galaxy S10 for example) run these shader modules happily.

#6

Hi @jgavris, I have found problem, Adreno driver cannot compile SPIR-V code for Compute Shader without spirv tools optimization. I compile hlsl code on Android using glslang tool without ENABLE_OPT macro.

#7

@AndreyOGL_D3D Thanks for the reply. Your message was a little unclear to me. Did you find it only works with or without spirv-opt?

#8

I use full code of glslang tool without ENABLE_OPT macro in 3DEngine project, so the final spirv code will not have optimizations.

This macro requried SPIRV optimization, so you must add spirv-tool project to glslang:

void GlslangToSpv(const TIntermediate& intermediate, std::vector<unsigned int>& spirv,
                  spv::SpvBuildLogger* logger, SpvOptions* options)
{
....
#if ENABLE_OPT
    // If from HLSL, run spirv-opt to "legalize" the SPIR-V for Vulkan
    // eg. forward and remove memory writes of opaque types.
    bool prelegalization = intermediate.getSource() == EShSourceHlsl;
    if ((intermediate.getSource() == EShSourceHlsl || options->optimizeSize) && !options->disableOptimizer) {
        SpirvToolsLegalize(intermediate, spirv, logger, options);
        prelegalization = false;
    }

    if (options->validate)
        SpirvToolsValidate(intermediate, spirv, logger, prelegalization);

    if (options->disassemble)
        SpirvToolsDisassemble(std::cout, spirv);

#endif

....
}

glslangvalidator.exe already has SPIRV optimizations option. I tried to use glslangvalidator.exe to produce spirv- code, in this case i have no problem!

Note!

  1. Adreno driver can compile spirv code without spirv-optimizations for Vertex/Pixel Shaders.
  2. Adreno driver cannot compile spirv code without spirv-optimizations for Compute Shaders

You can compare the ywo version of shaders:

without spirv-optimizations
with spirrv-optimizations

Do you have any questions ?

#9

That’s very helpful, thank you.

I’ve noticed that many of my compute shaders do compile and run fine on Adreno with optimizations though. It’s only one or two difficult ones, and I’ve yet to isolate what makes these ‘special’ compared to the others.

#10

@AndreyOGL_D3D Have you tried out Android Q yet? The shader compilation issue I had on Android P has magically gone away. I am also seeing substantial performance increases across the board.

#11

Hi! @jgavris

no, I have no device with Adreno/Android Q.

Good news! So, Qualcomm fixed Vulkan driver problem.
Right now I have another problem with Adreno, i see no 3D scene, only 2D text, but it work very well on Mali…