Tracking image layout transitions

Desperado17 · June 10, 2021, 8:28pm

I currently try to track down a bug where I assume a sampler (who is also an offscreen render target) is not in the expected layout when its layout descriptor is bound for rendering:

vkDebug: Validation: 0: Validation Error: [ VUID-VkDescriptorImageInfo-imageLayout-00344 ] Object 0: handle = 0x555557f9cc58, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0xde55a405 | vkCmdDraw(): Cannot use VkImage 0x1c000000001c[] (layer=0 mip=0) with specific layout VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL that doesn’t match the previous known layout VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL. The Vulkan spec states: imageLayout must match the actual VkImageLayout of each subresource accessible from imageView at the time this descriptor is accessed as defined by the image layout matching rules (https://vulkan.lunarg.com/doc/view/1.2.154.0/linux/1.2-extensions/vkspec.html#VUID-VkDescriptorImageInfo-imageLayout-00344)

I assume these parameters for the final transition are correct?

} else if ( oldLayout == VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL && newLayout == VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL) {
        barrier.srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT;
        barrier.dstAccessMask = VK_ACCESS_SHADER_READ_BIT;

        sourceStage = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT;
        destinationStage = VK_PIPELINE_STAGE_VERTEX_SHADER_BIT;
    } else if
     
<…>

    vkCmdPipelineBarrier(
        commandBuffer,
        sourceStage, destinationStage,
        0,
        0, nullptr,
        0, nullptr,
        1, &barrier
    );

Assuming the transition is correct and doesn’t fail, is there a validator or other way to track implicit layout transitions during runtime I may not be aware about?

Thanks for your help.

Alfonse_Reinheart · June 11, 2021, 12:05am

There is only one kind of “implicit” layout transition, and it’s not even particularly implicit. It’s just a different way of explicitly specifying layout transitions.

This “implicit” transition is the transition of layouts between subpasses. A render pass instance owns all images that are given to them as attachments, from the beginning of the render pass to its end. As such, you are not allowed to transition their layouts; the render pass system will do that.

But it will do it in accord with how you build the render pass. You specify what layout the image will be in when a render pass starts (it can be undefined if you’re going to clear or ignore the image’s current data). You specify what layout the render pass will leave the image in after all subpasses have completed. And each subpass specifies what layout the image will be transitioned to before that subpass starts doing its stuff.

It is up to the implementation to decide exactly when and where it will do those transitions. But it will do them before executing the subpass that uses a particular attachment in accord with the layout specified for that subpass.

I bring all of this up because your barrier makes no sense. Indeed, there is never a reason to have a barrier layout transition whose source is one of the attachment layouts. Attachment layouts are useful only for images that were being used as attachments.

If the image is no longer an attachment at the present time, then you must be after the render pass that previously used it as an attachment. So you may as well have had the render pass’s final layout to do the transition to the layout you’re trying to transition it to.

And if the image is currently being used as an attachment… you can’t change its layout. That image doesn’t belong to you anymore; it belongs to the render pass.

So clearly, something in your code has gone wrong.

Desperado17 · June 11, 2021, 6:41am

The intended use is that I leave the image in VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL until I bind it as an input texture at which point it will become VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL.

Otherwise, your post has info that I think includes the solution but I have to try first.

Desperado17 · June 11, 2021, 12:17pm

Is it right that I cannot use VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL in the imageLayout Parameter of VkDescriptorImageInfo during texture?

My problem is that I cannot use VkAttachmentDescription::finalLayout to convert the offscreen image to VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL directly at the end of the offscreen pass because in theory my engine flexibly allows more offscreen passes to the same texture before said texture is bound as sampler and used for reading.

What should I do? Keep the texture in VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL permanently then use VkAttachmentDescription::finalLayout = VkAttachmentDescription::initialLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL ?

krOoze · June 11, 2021, 1:43pm

The VkDescriptorImageInfo::imageLayout must be a layout that is appropriate for the Descriptor type, as the Valid Usage says.

OK. You can change the layout afterwards. But you must not mis-synchronize; i.e. you must form a dependency to whatever the previous usage of the image was when you decide to change the layout. Layout change is a write operation, so you must make sure any previous write or read already finished by then.

If you missynchronize, the layers will (currently) typically think the image is still in the old layout.

Desperado17 · June 11, 2021, 1:59pm

Then what is the recommended method for changing the layout of the offscreen texture last minute when I know it will be bound for texture fetch next? It will be decided at a point where vkCmdBeginRenderPass for the fetching render pass has already been called.
vkCmdPipelineBarrier is not possible during a render pass or only with restrictions.

Alfonse_Reinheart · June 11, 2021, 2:16pm

Don’t. There’s no reason to need to do this. What are you concerned about?

No, it was decided when you built your render pass, not when you begin the render pass. Beginning/ending the render pass instance merely makes it happen, but the layout is hard-coded into the render pass.

Desperado17 · June 11, 2021, 2:19pm

Are you saying I can/should bind a texture in state VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL as input texture?

Alfonse_Reinheart · June 11, 2021, 2:21pm

No. I’m saying that at the end of your render pass, you should transition the texture away from that layout to the layout you’re going to use it in. Which should always be the same layout.

And if it can’t be the same layout, then your render pass should pick the one that’s most likely to happen, and any code that needs a different layout can transition it themselves.

That is, you should establish a convention that, after the render pass, the attachment image will be in layout X (which is not the color attachment layout). So if you need layout Y, you know you need to transition it manually.

krOoze · June 11, 2021, 7:40pm

I lost track of what the problem is. I think it is much less brainfucky than you think it is.

If I understood correctly you have an image, and

You write some stuff to that image as an color attachment
You read the image through a samler

So you need to do three things:

make image COLOR_LAYOUT when it is written as attachment
make image SHADER_READ_LAYOUT when it is sampled (i.e. as non-attachment)
make sure the write finished before the read starts (resp. before the layout is changed). You do that by correct application of src and dst stage and access masks.

Desperado17 · June 11, 2021, 7:56pm

I might have a solution for the problems but I cannot try it before monday. Thanks for all the help so far!

Alfonse_Reinheart · June 11, 2021, 8:24pm

Which is best done as part of an external render pass dependency.

Desperado17 · June 14, 2021, 1:52pm

Ok, so I tried to perform the layout transition to attachment optimal directly before the render pass writing to the texture attachment starts. I get the following errors from the validation layer:

vkDebug: Validation: 0: Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x555558054b38, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | Submitted command buffer expects VkImage 0x1c000000001c[] (subresource: aspectMask 0x1 array layer 0, mip level 0) to be in layout VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL–instead, current layout is VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL.

vkDebug: Validation: 0: Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x555558054b38, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | Submitted command buffer expects VkImage 0x2a000000002a[] (subresource: aspectMask 0x1 array layer 0, mip level 0) to be in layout VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL–instead, current layout is VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL.

I stripped the application down to the minimum so only the main render targets are used.

In the primary buffer I do the following:

Render pass on image A that just performs a clear.
Layout transition to attachment for image B followed by render pass with clear on image B. Image should end up in read only optimal due to render pass finalLayout parameters.
Layout transition to attachment for image C followed by render pass with clear on image C. Image should end up in read only optimal due to render pass finalLayout parameters.
Image B and C are bound as input textures for another render pass to image A which draws a screen size quad.

I captured the api calls using nvidia nsight of one frame see next post.

Desperado17 · June 14, 2021, 1:56pm

The code regarding the transitions should be under “// Recorded 19 commands to primary command buffer”

ÔªøEvent,Description,CPU ms,GPU ms
0,// Start of Capture,-,-
1,"VkResult vkResetCommandBuffer(VkCommandBuffer commandBuffer = '0x0000558261927580', VkCommandBufferResetFlags flags = VkCommandBufferResetFlags(0)) = VK_SUCCESS",0.03,-
2,"VkResult vkWaitForFences(VkDevice device = '0x000055826140f038', uint32_t fenceCount = 1, const VkFence* pFences = '0x0000558261407118', VkBool32 waitAll = VK_TRUE, uint64_t timeout = 18446744073709551615) = VK_SUCCESS",0.02,-
3,"VkResult vkWaitForFences(VkDevice device = '0x000055826140f038', uint32_t fenceCount = 1, const VkFence* pFences = '0x0000558261407118', VkBool32 waitAll = VK_TRUE, uint64_t timeout = 18446744073709551615) = VK_SUCCESS",<0.01,-
4,"VkResult vkAllocateCommandBuffers(VkDevice device = '0x000055826140f038', const VkCommandBufferAllocateInfo* pAllocateInfo = { .sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO, .pNext = nullptr, .commandPool = '0x00005582619d9488', .level = VK_COMMAND_BUFFER_LEVEL_PRIMARY, .commandBufferCount = 1 }, VkCommandBuffer* pCommandBuffers = '0x0000558289784f50') = VK_SUCCESS",0.02,-
5,"// Recorded 3 commands to primary command buffer
6,"VkResult vkQueueSubmit(VkQueue queue = '0x000055826167a0e8', uint32_t submitCount = 1, const VkSubmitInfo* pSubmits = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO, .pNext = nullptr, .waitSemaphoreCount = 0, .pWaitSemaphores = nullptr, .pWaitDstStageMask = nullptr, .commandBufferCount = 1, .pCommandBuffers = '0x0000558289784f50', .signalSemaphoreCount = 0, .pSignalSemaphores = nullptr }, VkFence fence = 'VK_NULL_HANDLE') = VK_SUCCESS",0.06,-
7,// Primary command buffer 3 commands
8,"VkResult vkBeginCommandBuffer(VkCommandBuffer commandBuffer = '0x0000558289784f50', const VkCommandBufferBeginInfo* pBeginInfo = { .sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO, .pNext = nullptr, .flags = VkCommandBufferUsageFlags(VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT), .pInheritanceInfo = nullptr }) = VK_SUCCESS",-,-
9,"void vkCmdPipelineBarrier(VkCommandBuffer commandBuffer = '0x0000558289784f50', VkPipelineStageFlags srcStageMask = VkPipelineStageFlags(VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT), VkPipelineStageFlags dstStageMask = VkPipelineStageFlags(VK_PIPELINE_STAGE_VERTEX_SHADER_BIT), VkDependencyFlags dependencyFlags = VkDependencyFlags(0), uint32_t memoryBarrierCount = 0, const VkMemoryBarrier* pMemoryBarriers = nullptr, uint32_t bufferMemoryBarrierCount = 0, const VkBufferMemoryBarrier* pBufferMemoryBarriers = nullptr, uint32_t imageMemoryBarrierCount = 1, const VkImageMemoryBarrier* pImageMemoryBarriers = { .sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER, .pNext = nullptr, .srcAccessMask = VkAccessFlags(VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT), .dstAccessMask = VkAccessFlags(VK_ACCESS_SHADER_READ_BIT), .oldLayout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL, .newLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, .srcQueueFamilyIndex = 4294967295, .dstQueueFamilyIndex = 4294967295, .image = '0x00005582619d6968', .subresourceRange = { .aspectMask = VkImageAspectFlags(VK_IMAGE_ASPECT_COLOR_BIT), .baseMipLevel = 0, .levelCount = 1, .baseArrayLayer = 0, .layerCount = 1 } })",-,-
10,VkResult vkEndCommandBuffer(VkCommandBuffer commandBuffer = '0x0000558289784f50') = VK_SUCCESS,-,-
11,// End of queue submit,-,-
12,VkResult vkQueueWaitIdle(VkQueue queue = '0x000055826167a0e8') = VK_SUCCESS,0.15,-
13,"void vkFreeCommandBuffers(VkDevice device = '0x000055826140f038', VkCommandPool commandPool = '0x00005582619d9488', uint32_t commandBufferCount = 1, const VkCommandBuffer* pCommandBuffers = '0x0000558289784f50')",<0.01,-
14,"VkResult vkAllocateCommandBuffers(VkDevice device = '0x000055826140f038', const VkCommandBufferAllocateInfo* pAllocateInfo = { .sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO, .pNext = nullptr, .commandPool = '0x00005582619d9488', .level = VK_COMMAND_BUFFER_LEVEL_PRIMARY, .commandBufferCount = 1 }, VkCommandBuffer* pCommandBuffers = '0x0000558289526c00') = VK_SUCCESS",0.01,-
15,"// Recorded 3 commands to primary command buffer
16,"VkResult vkQueueSubmit(VkQueue queue = '0x000055826167a0e8', uint32_t submitCount = 1, const VkSubmitInfo* pSubmits = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO, .pNext = nullptr, .waitSemaphoreCount = 0, .pWaitSemaphores = nullptr, .pWaitDstStageMask = nullptr, .commandBufferCount = 1, .pCommandBuffers = '0x0000558289526c00', .signalSemaphoreCount = 0, .pSignalSemaphores = nullptr }, VkFence fence = 'VK_NULL_HANDLE') = VK_SUCCESS",0.03,-
17,// Primary command buffer 3 commands
18,"VkResult vkBeginCommandBuffer(VkCommandBuffer commandBuffer = '0x0000558289526c00', const VkCommandBufferBeginInfo* pBeginInfo = { .sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO, .pNext = nullptr, .flags = VkCommandBufferUsageFlags(VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT), .pInheritanceInfo = nullptr }) = VK_SUCCESS",-,-
19,"void vkCmdPipelineBarrier(VkCommandBuffer commandBuffer = '0x0000558289526c00', VkPipelineStageFlags srcStageMask = VkPipelineStageFlags(VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT), VkPipelineStageFlags dstStageMask = VkPipelineStageFlags(VK_PIPELINE_STAGE_VERTEX_SHADER_BIT), VkDependencyFlags dependencyFlags = VkDependencyFlags(0), uint32_t memoryBarrierCount = 0, const VkMemoryBarrier* pMemoryBarriers = nullptr, uint32_t bufferMemoryBarrierCount = 0, const VkBufferMemoryBarrier* pBufferMemoryBarriers = nullptr, uint32_t imageMemoryBarrierCount = 1, const VkImageMemoryBarrier* pImageMemoryBarriers = { .sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER, .pNext = nullptr, .srcAccessMask = VkAccessFlags(VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT), .dstAccessMask = VkAccessFlags(VK_ACCESS_SHADER_READ_BIT), .oldLayout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL, .newLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, .srcQueueFamilyIndex = 4294967295, .dstQueueFamilyIndex = 4294967295, .image = '0x00005582619f7b78', .subresourceRange = { .aspectMask = VkImageAspectFlags(VK_IMAGE_ASPECT_COLOR_BIT), .baseMipLevel = 0, .levelCount = 1, .baseArrayLayer = 0, .layerCount = 1 } })",-,-
20,VkResult vkEndCommandBuffer(VkCommandBuffer commandBuffer = '0x0000558289526c00') = VK_SUCCESS,-,-
21,// End of queue submit,-,-
22,VkResult vkQueueWaitIdle(VkQueue queue = '0x000055826167a0e8') = VK_SUCCESS,1.93,-
23,"void vkFreeCommandBuffers(VkDevice device = '0x000055826140f038', VkCommandPool commandPool = '0x00005582619d9488', uint32_t commandBufferCount = 1, const VkCommandBuffer* pCommandBuffers = '0x0000558289526c00')",<0.01,-
24,"VkResult vkCreateDescriptorSetLayout(VkDevice device = '0x000055826140f038', const VkDescriptorSetLayoutCreateInfo* pCreateInfo = { .sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO, .pNext = nullptr, .flags = VkDescriptorSetLayoutCreateFlags(0), .bindingCount = 2, .pBindings = { { .binding = 0, .descriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, .descriptorCount = 1, .stageFlags = VkShaderStageFlags(VK_SHADER_STAGE_FRAGMENT_BIT), .pImmutableSamplers = nullptr }, { .binding = 1, .descriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, .descriptorCount = 1, .stageFlags = VkShaderStageFlags(VK_SHADER_STAGE_FRAGMENT_BIT), .pImmutableSamplers = nullptr } } }, const VkAllocationCallbacks* pAllocator = nullptr, VkDescriptorSetLayout* pSetLayout = '0x000055828976aec8') = VK_SUCCESS",0.04,-
25,"VkResult vkCreateDescriptorSetLayout(VkDevice device = '0x000055826140f038', const VkDescriptorSetLayoutCreateInfo* pCreateInfo = { .sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO, .pNext = nullptr, .flags = VkDescriptorSetLayoutCreateFlags(0), .bindingCount = 0, .pBindings = nullptr }, const VkAllocationCallbacks* pAllocator = nullptr, VkDescriptorSetLayout* pSetLayout = '0x000055828757b478') = VK_SUCCESS",0.02,-
26,"VkResult vkCreatePipelineLayout(VkDevice device = '0x000055826140f038', const VkPipelineLayoutCreateInfo* pCreateInfo = { .sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO, .pNext = nullptr, .flags = VkPipelineLayoutCreateFlags(0), .setLayoutCount = 1, .pSetLayouts = '0x000055828976aec8', .pushConstantRangeCount = 0, .pPushConstantRanges = nullptr }, const VkAllocationCallbacks* pAllocator = nullptr, VkPipelineLayout* pPipelineLayout = '0x000055828a50e4b8') = VK_SUCCESS",0.05,-
27,"VkResult vkCreateGraphicsPipelines(VkDevice device = '0x000055826140f038', VkPipelineCache pipelineCache = 'VK_NULL_HANDLE', uint32_t createInfoCount = 1, const VkGraphicsPipelineCreateInfo* pCreateInfos = { .sType = VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO, .pNext = nullptr, .flags = VkPipelineCreateFlags(0), .stageCount = 2, .pStages = { { .sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO, .pNext = nullptr, .flags = VkPipelineShaderStageCreateFlags(0), .stage = VK_SHADER_STAGE_VERTEX_BIT, .module = '0x0000558262254898', .pName = main, .pSpecializationInfo = nullptr }, { .sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO, .pNext = nullptr, .flags = VkPipelineShaderStageCreateFlags(0), .stage = VK_SHADER_STAGE_FRAGMENT_BIT, .module = '0x000055826225dac8', .pName = main, .pSpecializationInfo = nullptr } }, .pVertexInputState = { .sType = VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO, .pNext = nullptr, .flags = VkPipelineVertexInputStateCreateFlags(0), .vertexBindingDescriptionCount = 1, .pVertexBindingDescriptions = { .binding = 0, .stride = 16, .inputRate = VK_VERTEX_INPUT_RATE_VERTEX }, .vertexAttributeDescriptionCount = 2, .pVertexAttributeDescriptions = { { .location = 0, .binding = 0, .format = VK_FORMAT_R32G32_SFLOAT, .offset = 0 }, { .location = 3, .binding = 0, .format = VK_FORMAT_R32G32_SFLOAT, .offset = 8 } } }, .pInputAssemblyState = { .sType = VK_STRUCTURE_TYPE_PIPELINE_INPUT_ASSEMBLY_STATE_CREATE_INFO, .pNext = nullptr, .flags = VkPipelineInputAssemblyStateCreateFlags(0), .topology = VK_PRIMITIVE_TOPOLOGY_TRIANGLE_STRIP, .primitiveRestartEnable = VK_FALSE }, .pTessellationState = nullptr, .pViewportState = { .sType = VK_STRUCTURE_TYPE_PIPELINE_VIEWPORT_STATE_CREATE_INFO, .pNext = nullptr, .flags = VkPipelineViewportStateCreateFlags(0), .viewportCount = 1, .pViewports = { .x = 0.0f, .y = 782.0f, .width = 982.0f, .height = -782.0f, .minDepth = 0.0f, .maxDepth = 1.0f }, .scissorCount = 1, .pScissors = { .offset = { .x = 0, .y = 0 }, .extent = { .width = 982, .height = 782 } } }, .pRasterizationState = { .sType = VK_STRUCTURE_TYPE_PIPELINE_RASTERIZATION_STATE_CREATE_INFO, .pNext = nullptr, .flags = VkPipelineRasterizationStateCreateFlags(0), .depthClampEnable = VK_FALSE, .rasterizerDiscardEnable = VK_FALSE, .polygonMode = VK_POLYGON_MODE_FILL, .cullMode = VkCullModeFlags(VK_CULL_MODE_BACK_BIT), .frontFace = VK_FRONT_FACE_COUNTER_CLOCKWISE, .depthBiasEnable = VK_FALSE, .depthBiasConstantFactor = 0.0f, .depthBiasClamp = 0.0f, .depthBiasSlopeFactor = 0.0f, .lineWidth = 1.0f }, .pMultisampleState = { .sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO, .pNext = nullptr, .flags = VkPipelineMultisampleStateCreateFlags(0), .rasterizationSamples = VK_SAMPLE_COUNT_1_BIT, .sampleShadingEnable = VK_FALSE, .minSampleShading = 0.0f, .pSampleMask = nullptr, .alphaToCoverageEnable = VK_FALSE, .alphaToOneEnable = VK_FALSE }, .pDepthStencilState = { .sType = VK_STRUCTURE_TYPE_PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO, .pNext = nullptr, .flags = VkPipelineDepthStencilStateCreateFlags(0), .depthTestEnable = VK_FALSE, .depthWriteEnable = VK_FALSE, .depthCompareOp = VK_COMPARE_OP_LESS, .depthBoundsTestEnable = VK_FALSE, .stencilTestEnable = VK_FALSE, .front = { .failOp = VK_STENCIL_OP_KEEP, .passOp = VK_STENCIL_OP_KEEP, .depthFailOp = VK_STENCIL_OP_KEEP, .compareOp = VK_COMPARE_OP_ALWAYS, .compareMask = 255, .writeMask = 4294967295, .reference = 0 }, .back = { .failOp = VK_STENCIL_OP_KEEP, .passOp = VK_STENCIL_OP_KEEP, .depthFailOp = VK_STENCIL_OP_KEEP, .compareOp = VK_COMPARE_OP_ALWAYS, .compareMask = 255, .writeMask = 4294967295, .reference = 0 }, .minDepthBounds = 0.0f, .maxDepthBounds = 0.0f }, .pColorBlendState = { .sType = VK_STRUCTURE_TYPE_PIPELINE_COLOR_BLEND_STATE_CREATE_INFO, .pNext = nullptr, .flags = VkPipelineColorBlendStateCreateFlags(0), .logicOpEnable = VK_FALSE, .logicOp = VK_LOGIC_OP_COPY, .attachmentCount = 1, .pAttachments = { .blendEnable = VK_TRUE, .srcColorBlendFactor = VK_BLEND_FACTOR_ONE, .dstColorBlendFactor = VK_BLEND_FACTOR_ZERO, .colorBlendOp = VK_BLEND_OP_ADD, .srcAlphaBlendFactor = VK_BLEND_FACTOR_ONE, .dstAlphaBlendFactor = VK_BLEND_FACTOR_ZERO, .alphaBlendOp = VK_BLEND_OP_ADD, .colorWriteMask = VkColorComponentFlags(VK_COLOR_COMPONENT_R_BIT | VK_COLOR_COMPONENT_G_BIT | VK_COLOR_COMPONENT_B_BIT | VK_COLOR_COMPONENT_A_BIT) }, .blendConstants = { 0.0f, 0.0f, 0.0f, 0.0f } }, .pDynamicState = nullptr, .layout = '0x000055828a50e4b8', .renderPass = '0x000055826191b7d8',  }, const VkAllocationCallbacks* pAllocator = nullptr, VkPipeline* pPipelines = '0x00005582897925f8') = VK_SUCCESS",0.17,-
28,"VkResult vkAllocateDescriptorSets(VkDevice device = '0x000055826140f038', const VkDescriptorSetAllocateInfo* pAllocateInfo = { .sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO, .pNext = nullptr, .descriptorPool = '0x0000558262255a18', .descriptorSetCount = 1, .pSetLayouts = '0x000055828976aec8' }, VkDescriptorSet* pDescriptorSets = '0x0000558264052ff0') = VK_SUCCESS",0.03,-
29,"void vkUpdateDescriptorSets(VkDevice device = '0x000055826140f038', uint32_t descriptorWriteCount = 1, const VkWriteDescriptorSet* pDescriptorWrites = { .sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET, .pNext = nullptr, .dstSet = '0x0000558264052ff0', .dstBinding = 0, .dstArrayElement = 0, .descriptorCount = 1, .descriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, .pImageInfo = { .sampler = '0x0000558261a09478', .imageView = '0x00005582619547b8', .imageLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL }, .pBufferInfo = nullptr, .pTexelBufferView = nullptr }, uint32_t descriptorCopyCount = 0, const VkCopyDescriptorSet* pDescriptorCopies = nullptr)",0.02,-
30,"void vkUpdateDescriptorSets(VkDevice device = '0x000055826140f038', uint32_t descriptorWriteCount = 1, const VkWriteDescriptorSet* pDescriptorWrites = { .sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET, .pNext = nullptr, .dstSet = '0x0000558264052ff0', .dstBinding = 1, .dstArrayElement = 0, .descriptorCount = 1, .descriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, .pImageInfo = { .sampler = '0x0000558262260ad8', .imageView = '0x00005582619fca38', .imageLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL }, .pBufferInfo = nullptr, .pTexelBufferView = nullptr }, uint32_t descriptorCopyCount = 0, const VkCopyDescriptorSet* pDescriptorCopies = nullptr)",<0.01,-
31,"// Recorded 19 commands to primary command buffer
32,"VkResult vkWaitForFences(VkDevice device = '0x000055826140f038', uint32_t fenceCount = 1, const VkFence* pFences = '0x0000558261407118', VkBool32 waitAll = VK_TRUE, uint64_t timeout = 18446744073709551615) = VK_SUCCESS",<0.01,-
33,"VkResult vkAcquireNextImageKHR(VkDevice device = '0x000055826140f038', VkSwapchainKHR swapchain = '0x00005582614b9b68', uint64_t timeout = 18446744073709551615, VkSemaphore semaphore = '0x0000558261940458', VkFence fence = 'VK_NULL_HANDLE', uint32_t* pImageIndex = 1) = VK_SUCCESS",0.03,-
34,"VkResult vkWaitForFences(VkDevice device = '0x000055826140f038', uint32_t fenceCount = 1, const VkFence* pFences = '0x0000558261407118', VkBool32 waitAll = VK_TRUE, uint64_t timeout = 18446744073709551615) = VK_SUCCESS",<0.01,-
35,"VkResult vkResetFences(VkDevice device = '0x000055826140f038', uint32_t fenceCount = 1, const VkFence* pFences = '0x0000558261407118') = VK_SUCCESS",<0.01,-
36,"VkResult vkQueueSubmit(VkQueue queue = '0x000055826167a0e8', uint32_t submitCount = 1, const VkSubmitInfo* pSubmits = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO, .pNext = nullptr, .waitSemaphoreCount = 1, .pWaitSemaphores = '0x0000558261940458', .pWaitDstStageMask = VkPipelineStageFlags(VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT), .commandBufferCount = 1, .pCommandBuffers = '0x0000558261927580', .signalSemaphoreCount = 1, .pSignalSemaphores = '0x0000558261940f28' }, VkFence fence = '0x0000558261407118') = VK_SUCCESS",0.03,-
37,// Primary command buffer 19 commands
38,"VkResult vkBeginCommandBuffer(VkCommandBuffer commandBuffer = '0x0000558261927580', const VkCommandBufferBeginInfo* pBeginInfo = { .sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO, .pNext = nullptr, .flags = VkCommandBufferUsageFlags(VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT), .pInheritanceInfo = nullptr }) = VK_SUCCESS",-,-
39,"void vkCmdBeginRenderPass(VkCommandBuffer commandBuffer = '0x0000558261927580', const VkRenderPassBeginInfo* pRenderPassBegin = { .sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO, .pNext = nullptr, .renderPass = '0x000055826191b7d8', .framebuffer = '0x0000558261920a78', .renderArea = { .offset = { .x = 0, .y = 0 }, .extent = { .width = 982, .height = 782 } }, .clearValueCount = 2, .pClearValues = { { .color = { .float32 = { 0.0f, 0.0f, 0.0f, 1.0f }, .int32 = { 0, 0, 0, 1065353216 }, .uint32 = { 0, 0, 0, 1065353216 } }, .depthStencil = { .depth = 0.0f, .stencil = 0 } }, { .color = { .float32 = { 1.0f, 0.0f, 0.0f, 0.0f }, .int32 = { 1065353216, 0, 0, 0 }, .uint32 = { 1065353216, 0, 0, 0 } }, .depthStencil = { .depth = 1.0f, .stencil = 0 } } } }, VkSubpassContents contents = VK_SUBPASS_CONTENTS_INLINE)",-,-
40,"void vkCmdClearAttachments(VkCommandBuffer commandBuffer = '0x0000558261927580', uint32_t attachmentCount = 1, const VkClearAttachment* pAttachments = { .aspectMask = VkImageAspectFlags(VK_IMAGE_ASPECT_COLOR_BIT), .colorAttachment = 0, .clearValue = { .color = { .float32 = { 0.7137255f, 0.88235295f, 0.98039216f, 1.0f }, .int32 = { 1060550327, 1063379426, 1065024251, 1065353216 }, .uint32 = { 1060550327, 1063379426, 1065024251, 1065353216 } }, .depthStencil = { .depth = 0.7137255f, .stencil = 1063379426 } } }, uint32_t rectCount = 1, const VkClearRect* pRects = { .rect = { .offset = { .x = 0, .y = 0 }, .extent = { .width = 982, .height = 782 } }, .baseArrayLayer = 0, .layerCount = 1 })",-,-
41,void vkCmdEndRenderPass(VkCommandBuffer commandBuffer = '0x0000558261927580'),-,-
42,"void vkCmdPipelineBarrier(VkCommandBuffer commandBuffer = '0x0000558261927580', VkPipelineStageFlags srcStageMask = VkPipelineStageFlags(VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT), VkPipelineStageFlags dstStageMask = VkPipelineStageFlags(VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT), VkDependencyFlags dependencyFlags = VkDependencyFlags(0), uint32_t memoryBarrierCount = 0, const VkMemoryBarrier* pMemoryBarriers = nullptr, uint32_t bufferMemoryBarrierCount = 0, const VkBufferMemoryBarrier* pBufferMemoryBarriers = nullptr, uint32_t imageMemoryBarrierCount = 1, const VkImageMemoryBarrier* pImageMemoryBarriers = { .sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER, .pNext = nullptr, .srcAccessMask = VkAccessFlags(VK_ACCESS_SHADER_READ_BIT), .dstAccessMask = VkAccessFlags(VK_ACCESS_COLOR_ATTACHMENT_READ_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT), .oldLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, .newLayout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL, .srcQueueFamilyIndex = 4294967295, .dstQueueFamilyIndex = 4294967295, .image = '0x00005582619d6968', .subresourceRange = { .aspectMask = VkImageAspectFlags(VK_IMAGE_ASPECT_COLOR_BIT), .baseMipLevel = 0, .levelCount = 1, .baseArrayLayer = 0, .layerCount = 1 } })",-,-
43,"void vkCmdBeginRenderPass(VkCommandBuffer commandBuffer = '0x0000558261927580', const VkRenderPassBeginInfo* pRenderPassBegin = { .sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO, .pNext = nullptr, .renderPass = '0x00005582619a6748', .framebuffer = '0x00005582619d5d28', .renderArea = { .offset = { .x = 0, .y = 0 }, .extent = { .width = 1000, .height = 800 } }, .clearValueCount = 1, .pClearValues = { .color = { .float32 = { 0.0f, 0.0f, 0.0f, 0.0f }, .int32 = { 0, 0, 0, 0 }, .uint32 = { 0, 0, 0, 0 } }, .depthStencil = { .depth = 0.0f, .stencil = 0 } } }, VkSubpassContents contents = VK_SUBPASS_CONTENTS_INLINE)",-,-
44,"void vkCmdClearAttachments(VkCommandBuffer commandBuffer = '0x0000558261927580', uint32_t attachmentCount = 1, const VkClearAttachment* pAttachments = { .aspectMask = VkImageAspectFlags(VK_IMAGE_ASPECT_COLOR_BIT), .colorAttachment = 0, .clearValue = { .color = { .float32 = { 0.7058824f, 0.8117647f, 0.9137255f, 1.0f }, .int32 = { 1060418741, 1062195152, 1063905770, 1065353216 }, .uint32 = { 1060418741, 1062195152, 1063905770, 1065353216 } }, .depthStencil = { .depth = 0.7058824f, .stencil = 1062195152 } } }, uint32_t rectCount = 1, const VkClearRect* pRects = { .rect = { .offset = { .x = 0, .y = 0 }, .extent = { .width = 1000, .height = 800 } }, .baseArrayLayer = 0, .layerCount = 1 })",-,-
45,void vkCmdEndRenderPass(VkCommandBuffer commandBuffer = '0x0000558261927580'),-,-
46,"void vkCmdPipelineBarrier(VkCommandBuffer commandBuffer = '0x0000558261927580', VkPipelineStageFlags srcStageMask = VkPipelineStageFlags(VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT), VkPipelineStageFlags dstStageMask = VkPipelineStageFlags(VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT), VkDependencyFlags dependencyFlags = VkDependencyFlags(0), uint32_t memoryBarrierCount = 0, const VkMemoryBarrier* pMemoryBarriers = nullptr, uint32_t bufferMemoryBarrierCount = 0, const VkBufferMemoryBarrier* pBufferMemoryBarriers = nullptr, uint32_t imageMemoryBarrierCount = 1, const VkImageMemoryBarrier* pImageMemoryBarriers = { .sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER, .pNext = nullptr, .srcAccessMask = VkAccessFlags(VK_ACCESS_SHADER_READ_BIT), .dstAccessMask = VkAccessFlags(VK_ACCESS_COLOR_ATTACHMENT_READ_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT), .oldLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, .newLayout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL, .srcQueueFamilyIndex = 4294967295, .dstQueueFamilyIndex = 4294967295, .image = '0x00005582619f7b78', .subresourceRange = { .aspectMask = VkImageAspectFlags(VK_IMAGE_ASPECT_COLOR_BIT), .baseMipLevel = 0, .levelCount = 1, .baseArrayLayer = 0, .layerCount = 1 } })",-,-
47,"void vkCmdBeginRenderPass(VkCommandBuffer commandBuffer = '0x0000558261927580', const VkRenderPassBeginInfo* pRenderPassBegin = { .sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO, .pNext = nullptr, .renderPass = '0x00005582619fd568', .framebuffer = '0x00005582619fe368', .renderArea = { .offset = { .x = 0, .y = 0 }, .extent = { .width = 1000, .height = 800 } }, .clearValueCount = 1, .pClearValues = { .color = { .float32 = { 0.0f, 0.0f, 0.0f, 0.0f }, .int32 = { 0, 0, 0, 0 }, .uint32 = { 0, 0, 0, 0 } }, .depthStencil = { .depth = 0.0f, .stencil = 0 } } }, VkSubpassContents contents = VK_SUBPASS_CONTENTS_INLINE)",-,-
48,"void vkCmdClearAttachments(VkCommandBuffer commandBuffer = '0x0000558261927580', uint32_t attachmentCount = 1, const VkClearAttachment* pAttachments = { .aspectMask = VkImageAspectFlags(VK_IMAGE_ASPECT_COLOR_BIT), .colorAttachment = 0, .clearValue = { .color = { .float32 = { 0.0f, 0.0f, 0.0f, 0.0f }, .int32 = { 0, 0, 0, 0 }, .uint32 = { 0, 0, 0, 0 } }, .depthStencil = { .depth = 0.0f, .stencil = 0 } } }, uint32_t rectCount = 1, const VkClearRect* pRects = { .rect = { .offset = { .x = 0, .y = 0 }, .extent = { .width = 1000, .height = 800 } }, .baseArrayLayer = 0, .layerCount = 1 })",-,-
49,void vkCmdEndRenderPass(VkCommandBuffer commandBuffer = '0x0000558261927580'),-,-
50,"void vkCmdBeginRenderPass(VkCommandBuffer commandBuffer = '0x0000558261927580', const VkRenderPassBeginInfo* pRenderPassBegin = { .sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO, .pNext = nullptr, .renderPass = '0x000055826191b7d8', .framebuffer = '0x0000558261920a78', .renderArea = { .offset = { .x = 0, .y = 0 }, .extent = { .width = 982, .height = 782 } }, .clearValueCount = 2, .pClearValues = { { .color = { .float32 = { 0.0f, 0.0f, 0.0f, 1.0f }, .int32 = { 0, 0, 0, 1065353216 }, .uint32 = { 0, 0, 0, 1065353216 } }, .depthStencil = { .depth = 0.0f, .stencil = 0 } }, { .color = { .float32 = { 1.0f, 0.0f, 0.0f, 0.0f }, .int32 = { 1065353216, 0, 0, 0 }, .uint32 = { 1065353216, 0, 0, 0 } }, .depthStencil = { .depth = 1.0f, .stencil = 0 } } } }, VkSubpassContents contents = VK_SUBPASS_CONTENTS_INLINE)",-,-
51,"void vkCmdBindPipeline(VkCommandBuffer commandBuffer = '0x0000558261927580', VkPipelineBindPoint pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS, VkPipeline pipeline = '0x00005582897925f8')",-,-
52,"void vkCmdBindVertexBuffers(VkCommandBuffer commandBuffer = '0x0000558261927580', uint32_t firstBinding = 0, uint32_t bindingCount = 1, const VkBuffer* pBuffers = '0x000055828751a198', const VkDeviceSize* pOffsets = 0)",-,-
53,"void vkCmdBindDescriptorSets(VkCommandBuffer commandBuffer = '0x0000558261927580', VkPipelineBindPoint pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS, VkPipelineLayout layout = '0x000055828a50e4b8', uint32_t firstSet = 0, uint32_t descriptorSetCount = 1, const VkDescriptorSet* pDescriptorSets = '0x0000558264052ff0', uint32_t dynamicOffsetCount = 0, const uint32_t* pDynamicOffsets = nullptr)",-,-
54,"void vkCmdDraw(VkCommandBuffer commandBuffer = '0x0000558261927580', uint32_t vertexCount = 4, uint32_t instanceCount = 1, uint32_t firstVertex = 0, uint32_t firstInstance = 0)",-,0.11
55,void vkCmdEndRenderPass(VkCommandBuffer commandBuffer = '0x0000558261927580'),-,-
56,VkResult vkEndCommandBuffer(VkCommandBuffer commandBuffer = '0x0000558261927580') = VK_SUCCESS,-,-
57,// End of queue submit,-,-
58,"VkResult vkQueuePresentKHR(VkQueue queue = '0x000055826167a0e8', const VkPresentInfoKHR* pPresentInfo = { .sType = VK_STRUCTURE_TYPE_PRESENT_INFO_KHR, .pNext = nullptr, .waitSemaphoreCount = 1, .pWaitSemaphores = '0x0000558261940f28', .swapchainCount = 1, .pSwapchains = '0x00005582614b9b68', .pImageIndices = 1, .pResults = nullptr }) = VK_SUCCESS",<0.01,-

krOoze · June 14, 2021, 2:06pm

API dump, perfect! For clarity, could you mark at which point the validation errors are reported?

Desperado17 · June 14, 2021, 2:10pm

I don’t know the vkDebug message doesn’t seem to tell it and I don’t know if Nsight can tell.

Desperado17 · June 14, 2021, 2:19pm

Wait, there is a barrier I cannot explain at the moment…

Alfonse_Reinheart · June 14, 2021, 2:19pm

This API dump is… weird. Is this from multiple threads without ordering between them or something? Because there is so much about this dump that just doesn’t make sense if it all happens in order.

The biggest (but by no means only) example is that you begin a render pass, clear an attachment (already redundant, since you should have cleared it on load), and then immediately end it without doing anything in the render pass. That you do this at all is already dubious, but the fact that you do it three times in a row just feels nuts.

This has to be something interleaved between multiple threads, right?

Desperado17 · June 14, 2021, 2:30pm

No, that’s all one thread. The reason why it looks so plain is that I removed everything useful that might distract. Otherwise, it might just be the way Nsight outputs events.

Please wait a moment before looking at it btw I think I found something.

krOoze · June 14, 2021, 2:38pm

You can simply run your app with this env:

set VK_INSTANCE_LAYERS=VK_LAYER_LUNARG_api_dump;VK_LAYER_KHRONOS_validation

(layers in this order). It should produce log in stdout with apidump and validation errors interposed. Then run your app with app > log.txt, if you need to redirect to file.