Hazard WRITE_AFTER_WRITE

wusikijeronii · March 9, 2024, 2:22pm

Hello. If I run Vulkan Configurator in Syncronization mode, I get the error:

ERROR Validation Error: [ SYNC-HAZARD-WRITE-AFTER-WRITE ] Object 0: handle = 0xb097c90000000027, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0x5c0ec5d6 | vkCmdEndRenderPass: Hazard WRITE_AFTER_WRITE in subpass 2 for attachment 1 color aspect during store with storeOp VK_ATTACHMENT_STORE_OP_STORE. Access info (usage: SYNC_COLOR_ATTACHMENT_OUTPUT_COLOR_ATTACHMENT_WRITE, prior_usage: SYNC_IMAGE_LAYOUT_TRANSITION, write_barriers: SYNC_FRAGMENT_SHADER_INPUT_ATTACHMENT_READ|SYNC_FRAGMENT_SHADER_SHADER_SAMPLED_READ|SYNC_FRAGMENT_SHADER_SHADER_STORAGE_READ|SYNC_FRAGMENT_SHADER_UNIFORM_READ, command: vkCmdNextSubpass, seq_no: 5, subcmd: 1,renderpass: VkRenderPass 0xb097c90000000027, reset_no: 1)

I have 3 subpasses:
0: scene
1: UI
2: Composition subpass. It takes image results from 0 and 1 subpasses.

My render pass creation function:

void Swapchain::createRenderPass()
{
    vk::AttachmentDescription mainpassAttachment;
    mainpassAttachment.setFormat(_imageFormat)
        .setSamples(vk::SampleCountFlagBits::e1)
        .setLoadOp(vk::AttachmentLoadOp::eClear)
        .setStoreOp(vk::AttachmentStoreOp::eStore)
        .setStencilLoadOp(vk::AttachmentLoadOp::eDontCare)
        .setStencilStoreOp(vk::AttachmentStoreOp::eDontCare)
        .setInitialLayout(vk::ImageLayout::eUndefined)
        .setFinalLayout(vk::ImageLayout::eShaderReadOnlyOptimal);
    vk::AttachmentReference mainpassAttachmentRef(0, vk::ImageLayout::eColorAttachmentOptimal);

    vk::AttachmentDescription overlayAttachment;
    overlayAttachment.setFormat(_imageFormat)
        .setSamples(vk::SampleCountFlagBits::e1)
        .setLoadOp(vk::AttachmentLoadOp::eClear)
        .setStoreOp(vk::AttachmentStoreOp::eStore)
        .setStencilLoadOp(vk::AttachmentLoadOp::eDontCare)
        .setStencilStoreOp(vk::AttachmentStoreOp::eDontCare)
        .setInitialLayout(vk::ImageLayout::eUndefined)
        .setFinalLayout(vk::ImageLayout::eShaderReadOnlyOptimal);
    vk::AttachmentReference overlayAttachmentRef(1, vk::ImageLayout::eColorAttachmentOptimal);

    vk::AttachmentDescription colorAttachment;
    colorAttachment.setFormat(_imageFormat)
        .setSamples(vk::SampleCountFlagBits::e1)
        .setLoadOp(vk::AttachmentLoadOp::eClear)
        .setStoreOp(vk::AttachmentStoreOp::eStore)
        .setStencilLoadOp(vk::AttachmentLoadOp::eDontCare)
        .setStencilStoreOp(vk::AttachmentStoreOp::eDontCare)
        .setInitialLayout(vk::ImageLayout::eUndefined)
        .setFinalLayout(vk::ImageLayout::ePresentSrcKHR);
    vk::AttachmentReference colorAttachmentRef(2, vk::ImageLayout::eColorAttachmentOptimal);

    vk::AttachmentDescription depthAttachment;
    depthAttachment.setFormat(_depthFormat)
        .setSamples(vk::SampleCountFlagBits::e1)
        .setLoadOp(vk::AttachmentLoadOp::eClear)
        .setStoreOp(vk::AttachmentStoreOp::eDontCare)
        .setStencilLoadOp(vk::AttachmentLoadOp::eDontCare)
        .setStencilStoreOp(vk::AttachmentStoreOp::eDontCare)
        .setInitialLayout(vk::ImageLayout::eUndefined)
        .setFinalLayout(vk::ImageLayout::eDepthStencilAttachmentOptimal);
    vk::AttachmentReference depthAttachmentRef(3, vk::ImageLayout::eDepthStencilAttachmentOptimal);

    vk::AttachmentReference unusedAttachmentRef;
    unusedAttachmentRef.attachment = vk::AttachmentUnused;
    unusedAttachmentRef.layout = vk::ImageLayout::eUndefined;

    std::array<vk::SubpassDescription, 3> subpasses;

    subpasses[0]
        .setPipelineBindPoint(vk::PipelineBindPoint::eGraphics)
        .setColorAttachmentCount(1)
        .setPColorAttachments(&mainpassAttachmentRef)
        .setPDepthStencilAttachment(&depthAttachmentRef);

    subpasses[1]
        .setPipelineBindPoint(vk::PipelineBindPoint::eGraphics)
        .setColorAttachmentCount(1)
        .setPColorAttachments(&overlayAttachmentRef)
        .setPDepthStencilAttachment(&unusedAttachmentRef);

    std::array<vk::AttachmentReference, 2> inputAttachments;
    inputAttachments[0] = vk::AttachmentReference(0, vk::ImageLayout::eShaderReadOnlyOptimal);
    inputAttachments[1] = vk::AttachmentReference(1, vk::ImageLayout::eShaderReadOnlyOptimal);
    subpasses[2]
        .setPipelineBindPoint(vk::PipelineBindPoint::eGraphics)
        .setInputAttachmentCount(2)
        .setPInputAttachments(inputAttachments.data())
        .setColorAttachmentCount(1)
        .setPColorAttachments(&colorAttachmentRef)
        .setPDepthStencilAttachment(&unusedAttachmentRef);

    std::array<vk::SubpassDependency, 4> dependencies;

    dependencies[0]
        .setSrcSubpass(VK_SUBPASS_EXTERNAL)
        .setDstSubpass(0)
        .setSrcStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setDstStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput |
                         vk::PipelineStageFlagBits::eEarlyFragmentTests)
        .setSrcAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDstAccessMask(vk::AccessFlagBits::eColorAttachmentRead | vk::AccessFlagBits::eShaderRead |
                          vk::AccessFlagBits::eColorAttachmentWrite | vk::AccessFlagBits::eDepthStencilAttachmentWrite)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

    dependencies[1]
        .setSrcSubpass(0)
        .setDstSubpass(1)
        .setSrcStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setDstStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setSrcAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDstAccessMask(vk::AccessFlagBits::eShaderRead | vk::AccessFlagBits::eInputAttachmentRead |
                          vk::AccessFlagBits::eColorAttachmentWrite)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

    dependencies[2]
        .setSrcSubpass(1)
        .setDstSubpass(2)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setDstStageMask(vk::PipelineStageFlagBits::eFragmentShader)
        .setSrcAccessMask(vk::AccessFlagBits::eColorAttachmentWrite)
        .setDstAccessMask(vk::AccessFlagBits::eShaderRead | vk::AccessFlagBits::eInputAttachmentRead)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

    dependencies[3]
        .setSrcSubpass(2)
        .setDstSubpass(VK_SUBPASS_EXTERNAL)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setDstStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setSrcAccessMask(vk::AccessFlagBits::eColorAttachmentRead | vk::AccessFlagBits::eColorAttachmentWrite)
        .setDstAccessMask(vk::AccessFlagBits::eMemoryRead);

    std::array<vk::AttachmentDescription, 4> attachments = {mainpassAttachment, overlayAttachment, colorAttachment,
                                                            depthAttachment};
    vk::RenderPassCreateInfo renderPassInfo;
    renderPassInfo.setAttachmentCount(static_cast<u32>(attachments.size()))
        .setPAttachments(attachments.data())
        .setSubpassCount(3)
        .setPSubpasses(subpasses.data())
        .setDependencyCount(dependencies.size())
        .setPDependencies(dependencies.data());

    if (_device.vkDevice().createRenderPass(&renderPassInfo, nullptr, &_renderPass, _device.vkLoader()) !=
        vk::Result::eSuccess)
        throw std::runtime_error("Failed to create render pass");
}

I tried to experiment with different masks in dependencies, but I still get the error. How can I solve this issue? Which dependency parameters should I use?

UPD: I cloned Sasha Williams examples, and I’ve faced the same errors when I try to launch ./build/bin/inputattachments.exe:

UNASSIGNED-CreateInstance-status-message(INFO / SPEC): msgNum: -2016116905 - Validation Information: [ UNASSIGNED-CreateInstance-status-message ] Object 0: handle = 0x1e4330c4bc0, type = VK_OBJECT_TYPE_INSTANCE; | MessageID = 0x87d47f57 | Khronos Validation Layer Active:
    Settings File: Found at C:\Users\alekesej\AppData\Local\LunarG\vkconfig\override\vk_layer_settings.txt specified by VkConfig application override.
    Current Enables: VK_VALIDATION_FEATURE_ENABLE_SYNCHRONIZATION_VALIDATION, VALIDATION_CHECK_ENABLE_SYNCHRONIZATION_VALIDATION_QUEUE_SUBMIT.
    Current Disables: None.

    Objects: 1
        [0] 0x1e4330c4bc0, type: 1, name: NULL
SYNC-HAZARD-WRITE-AFTER-WRITE(ERROR / SPEC): msgNum: 1544472022 - Validation Error: [ SYNC-HAZARD-WRITE-AFTER-WRITE ] Object 0: handle = 0x3a6cbb0000000025, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0x5c0ec5d6 | vkCmdNextSubpass: Hazard WRITE_AFTER_WRITE in subpass 1 for attachment 2 image layout transition (old_layout: VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL, new_layout: VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL). Access info (usage: SYNC_IMAGE_LAYOUT_TRANSITION, prior_usage: SYNC_LATE_FRAGMENT_TESTS_DEPTH_STENCIL_ATTACHMENT_WRITE, write_barriers: 0, command: vkCmdDrawIndexed, seq_no: 87, reset_no: 1).
    Objects: 1
        [0] 0x3a6cbb0000000025, type: 18, name: NULL
SYNC-HAZARD-READ-AFTER-WRITE(ERROR / SPEC): msgNum: -455515022 - Validation Error: [ SYNC-HAZARD-READ-AFTER-WRITE ] Object 0: handle = 0xd10d270000000018, type = VK_OBJECT_TYPE_IMAGE_VIEW; | MessageID = 0xe4d96472 | vkCmdDraw: Hazard READ_AFTER_WRITE for VkImageView 0xd10d270000000018[], in VkCommandBuffer 0x1e43f5264f0[], and VkPipeline 0xd84b930000000126[], VkDescriptorSet 0x32749b000000011e[], type: VK_DESCRIPTOR_TYPE_INPUT_ATTACHMENT, imageLayout: VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, binding #1, index 0. Access info (usage: SYNC_FRAGMENT_SHADER_INPUT_ATTACHMENT_READ, prior_usage: SYNC_IMAGE_LAYOUT_TRANSITION, write_barriers: SYNC_FRAGMENT_SHADER_SHADER_SAMPLED_READ|SYNC_FRAGMENT_SHADER_SHADER_STORAGE_READ|SYNC_FRAGMENT_SHADER_UNIFORM_READ, command: vkCmdNextSubpass, seq_no: 88, subcmd: 1,renderpass: VkRenderPass 0x3a6cbb0000000025[], reset_no: 1).
    Objects: 1
        [0] 0xd10d270000000018, type: 14, name: NULL

krOoze · March 10, 2024, 9:32pm

Might be a bug that the layers can’t detect the nill layout transition (which shouldn’t need sync). Make sure to update your layers to the latest.

Also your sync strategy is kinda non-obvious. Try to directly add dependency between External->subpass nr. 1 (where attachment 1 is loaded\cleared), it give it VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT+VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT. That might clear the error.

Otherwisely:

Everything is Undefined (and subsequently cleared). So I am not entirely sure what the eBottomOfPipe is intended for in the first dependency.
eBottomOfPipe has no accesses. The appropriate access flags are 0 resp. VK_ACCESS_NONE.
There seem to be some eShaderRead sprinkled in there, but not clear what for. In some cases it doesn’t even seem like valid usage (it has no compatible stage).
Whole dependencies[1] confuses me a bit what it’s trying to achieve.
The Clear and Store Ops are all Writes. Some of those Read access flags seem redundant.
Attachment nr. 0 (“mainpassAttachment”) seem to be unused between uses, but not present in Preserve attachments.

wusikijeronii · March 11, 2024, 8:41am

Thanks for your reply.
I updated the Vulkan SDK but then started getting more error messages.
Also, I updated my depedicies due to your remarks:

    dependencies[0]
        .setSrcSubpass(vk::SubpassExternal)
        .setDstSubpass(0)
        .setSrcStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setDstStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput |
                         vk::PipelineStageFlagBits::eEarlyFragmentTests)
        .setSrcAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDstAccessMask(vk::AccessFlagBits::eColorAttachmentWrite | vk::AccessFlagBits::eDepthStencilAttachmentWrite);

    dependencies[1]
        .setSrcSubpass(vk::SubpassExternal)
        .setDstSubpass(1)
        .setSrcStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setDstStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setSrcAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDstAccessMask(vk::AccessFlagBits::eColorAttachmentWrite);

    dependencies[2]
        .setSrcSubpass(0)
        .setDstSubpass(2)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setDstStageMask(vk::PipelineStageFlagBits::eFragmentShader)
        .setSrcAccessMask(vk::AccessFlagBits::eColorAttachmentWrite)
        .setDstAccessMask(vk::AccessFlagBits::eColorAttachmentRead)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

    dependencies[3]
        .setSrcSubpass(1)
        .setDstSubpass(2)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setDstStageMask(vk::PipelineStageFlagBits::eFragmentShader)
        .setSrcAccessMask(vk::AccessFlagBits::eColorAttachmentWrite)
        .setDstAccessMask(vk::AccessFlagBits::eColorAttachmentRead)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

    dependencies[4]
        .setSrcSubpass(2)
        .setDstSubpass(vk::SubpassExternal)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setDstStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setSrcAccessMask(vk::AccessFlagBits::eColorAttachmentRead | vk::AccessFlagBits::eColorAttachmentWrite)
        .setDstAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

Description:

Dependency 0: From external to subpass 0. 'cos src was external, SrcStageMask is eBottomOfPipe. Here we write color and depth attachments. So then I use eColorAttachmentOutput and eEarlyFragmentTests.
Dependency 1: The subpass doesn’t depend on anything. So then I mark it as going from external to subpass 1. Masks are the same as in Dependency 0, but without depth.
Dependency 2: Here I mark Subpass have to wait until color attachment will be rendered and available in fragment shader as input attachment. So then I set stage mask as eColorAttachmentOutput to eFragmentShader and access mask as eColorAttachmentWrite to eColorAttachmentRead.
Dependency 3: Same as the previous dependency
Dependency 4: Render the final image and return the control to the world. Here we read input attachments and write the output image. I set the stage mask as eColorAttachmentOutput to eBottomOfPipe (external) and the access mask as eColorAttachmentRead and eColorAttachmentWrite to eMemoryRead (read output image).

Is it correct, right?

New errors:
My app:

ERROR   Validation Error: [ SYNC-HAZARD-WRITE-AFTER-WRITE ] Object 0: handle = 0x4295ab0000000035, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0x5c0ec5d6 | vkCmdEndRenderPass():  Hazard WRITE_AFTER_WRITE in subpass 2 for attachment 1 color aspect during store with storeOp VK_ATTACHMENT_STORE_OP_STORE. Access info (usage: SYNC_COLOR_ATTACHMENT_OUTPUT_COLOR_ATTACHMENT_WRITE, prior_usage: SYNC_IMAGE_LAYOUT_TRANSITION, write_barriers: SYNC_FRAGMENT_SHADER_COLOR_ATTACHMENT_READ, command: vkCmdNextSubpass, seq_no: 5, subcmd: 1, renderpass: VkRenderPass 0x4295ab0000000035[], reset_no: 1)
ERROR   Validation Error: [ SYNC-HAZARD-WRITE-AFTER-READ ] Object 0: handle = 0x2aa9de25c00, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x376bc9df | vkQueueSubmit():  Hazard WRITE_AFTER_READ for entry 0, VkCommandBuffer 0x2aa9ea2ef10[], Submitted access info (submitted_usage: SYNC_IMAGE_LAYOUT_TRANSITION, command: vkCmdNextSubpass, seq_no: 5, subcmd: 1, renderpass: VkRenderPass 0x4295ab0000000035[], reset_no: 1). Access info (prior_usage: SYNC_PRESENT_ENGINE_SYNCVAL_PRESENT_ACQUIRE_READ_SYNCVAL, read_barriers: VK_PIPELINE_STAGE_2_FRAGMENT_SHADER_BIT|VK_PIPELINE_STAGE_2_EARLY_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_2_LATE_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_2_COLOR_ATTACHMENT_OUTPUT_BIT|VK_PIPELINE_STAGE_2_BOTTOM_OF_PIPE_BIT, , batch_tag: 96, vkAcquireNextImageKHR aquire_tag:96: VkSwapchainKHR 0x9fde6b0000000014[], image_index: 0image: VkImage 0xdd3a8a0000000015[]).

Sasha’s example:

WARNING-CreateInstance-status-message(INFO / SPEC): msgNum: 601872502 - Validation Information: [ WARNING-CreateInstance-status-message ] Object 0: handle = 0x1f401d26ef0, type = VK_OBJECT_TYPE_INSTANCE; | MessageID = 0x23dfd876 | vkCreateInstance():  Khronos Validation Layer Active:
    Settings File: Found at C:\Users\alekesej\AppData\Local\LunarG\vkconfig\override\vk_layer_settings.txt specified by VkConfig application override.
    Current Enables: VK_VALIDATION_FEATURE_ENABLE_SYNCHRONIZATION_VALIDATION.
    Current Disables: None.

    Objects: 1
        [0] 0x1f401d26ef0, type: 1, name: NULL
SYNC-HAZARD-WRITE-AFTER-WRITE(ERROR / SPEC): msgNum: 1544472022 - Validation Error: [ SYNC-HAZARD-WRITE-AFTER-WRITE ] Object 0: handle = 0x3a6cbb0000000025, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0x5c0ec5d6 | vkCmdNextSubpass():  Hazard WRITE_AFTER_WRITE in subpass 1 for attachment 2 image layout transition (old_layout: VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL, new_layout: VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL). Access info (usage: SYNC_IMAGE_LAYOUT_TRANSITION, prior_usage: SYNC_LATE_FRAGMENT_TESTS_DEPTH_STENCIL_ATTACHMENT_WRITE, write_barriers: 0, command: vkCmdDrawIndexed, seq_no: 87, reset_no: 1).
    Objects: 1
        [0] 0x3a6cbb0000000025, type: 18, name: NULL
SYNC-HAZARD-READ-AFTER-WRITE(ERROR / SPEC): msgNum: -455515022 - Validation Error: [ SYNC-HAZARD-READ-AFTER-WRITE ] Object 0: handle = 0xd10d270000000018, type = VK_OBJECT_TYPE_IMAGE_VIEW; | MessageID = 0xe4d96472 | vkCmdDraw():  Hazard READ_AFTER_WRITE for VkImageView 0xd10d270000000018[], in VkCommandBuffer 0x1f4109388c0[], and VkPipeline 0xd84b930000000126[], VkDescriptorSet 0x32749b000000011e[], type: VK_DESCRIPTOR_TYPE_INPUT_ATTACHMENT, imageLayout: VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, binding #1, index 0. Access info (usage: SYNC_FRAGMENT_SHADER_INPUT_ATTACHMENT_READ, prior_usage: SYNC_IMAGE_LAYOUT_TRANSITION, write_barriers: SYNC_FRAGMENT_SHADER_SHADER_SAMPLED_READ|SYNC_FRAGMENT_SHADER_SHADER_STORAGE_READ|SYNC_FRAGMENT_SHADER_UNIFORM_READ, command: vkCmdNextSubpass, seq_no: 88, subcmd: 1, renderpass: VkRenderPass 0x3a6cbb0000000025[], reset_no: 1).
    Objects: 1
        [0] 0xd10d270000000018, type: 14, name: NULL
SYNC-HAZARD-WRITE-AFTER-READ(ERROR / SPEC): msgNum: 929810911 - Validation Error: [ SYNC-HAZARD-WRITE-AFTER-READ ] Object 0: handle = 0x1f4058682e0, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x376bc9df | vkQueueSubmit():  Hazard WRITE_AFTER_READ for entry 0, VkCommandBuffer 0x1f4109388c0[], Submitted access info (submitted_usage: SYNC_IMAGE_LAYOUT_TRANSITION, command: vkCmdNextSubpass, seq_no: 88, subcmd: 1, renderpass: VkRenderPass 0x3a6cbb0000000025[], reset_no: 1). Access info (prior_usage: SYNC_PRESENT_ENGINE_SYNCVAL_PRESENT_ACQUIRE_READ_SYNCVAL, read_barriers: VK_PIPELINE_STAGE_2_FRAGMENT_SHADER_BIT|VK_PIPELINE_STAGE_2_EARLY_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_2_LATE_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_2_COLOR_ATTACHMENT_OUTPUT_BIT|VK_PIPELINE_STAGE_2_BOTTOM_OF_PIPE_BIT, , batch_tag: 12, vkAcquireNextImageKHR aquire_tag:12: VkSwapchainKHR 0xf443490000000006[], image_index: 0image: VkImage 0xcb3ee80000000007[]).
    Objects: 1
        [0] 0x1f4058682e0, type: 4, name: NULL

About your notes:
I’d like to emphasize that I tried to experiment with different flags in masks to resolve the issue. So in some places, I can have some unnecessary or irrelevant flag bits.

For from/to VK_SUBPASS_EXTERNAL I set eBottomOfPipe. If I set 0, then I get the error:

ERROR   Validation Error: [ VUID-VkSubpassDependency-srcStageMask-03937 ] | MessageID = 0x8150b373 | vkCreateRenderPass(): pDependencies[0].srcStageMask must not be 0 unless synchronization2 is enabled. The Vulkan spec states: If the synchronization2 feature is not enabled, srcStageMask must not be 0 (https://vulkan.lunarg.com/doc/view/1.3.275.0/windows/1.3-extensions/vkspec.html#VUID-VkSubpassDependency-srcStageMask-03937)

It’s just an artifact of my experiments. I thought it might help me solve the issue.

Yes, I agree. It’s incorrect. When I got the sync warning, I tried to reach better sync. So then I try to mark subass 1, which will depend on subpass 0.

To be honest, I don’t understand what you mean. ‘mainpassAttachment’ is the output attachment of subpass 0 and the input attachment in subpass 2.

Let’s conclude.
The issue can be either a Vulkan Sync Validation Layer bug depending on my OS, Envidroment, etc., or it could actually be an error in my code. I think it is the first option, but I am not sure exactly. Tomorrow I can check on another machine on Linux (now I use Windows). But in any case, I will want to ask: Can you reproduce this issue? Can you run any project with multisubpass rendering? At least, can you clone Sascha Willems examples and try to launch the “inputattachments” example?

krOoze · March 11, 2024, 12:35pm

wusikijeronii:

    dependencies[2]
        .setSrcSubpass(0)
        .setDstSubpass(2)
        .setDstStageMask(vk::PipelineStageFlagBits::eFragmentShader)
        .setDstAccessMask(vk::AccessFlagBits::eColorAttachmentRead)

    dependencies[3]
        .setSrcSubpass(1)
        .setDstSubpass(2)
        .setDstStageMask(vk::PipelineStageFlagBits::eFragmentShader)
        .setDstAccessMask(vk::AccessFlagBits::eColorAttachmentRead)

This seems incorrect. Did you mean eInputAttachmentRead?

Seems it now doesn’t like how the vkAcquireNextImage is synchronized. Do you wait on the semaphore?

They are not often updated, and they were created in the dark ages of Vulkan. Validation errors are unfortunately not uncommon there. I will give it a look later.

There’s nothing wrong with eBottomOfPipe, if that is intended. It means “block everything”. If you want to block nothing, then that is src=TOP in Vulkan 1.0. Most likely this src should chain with a semaphore and its pWaitDstStage though.

Neither BOTTOM nor TOP has accesses. The appropriate access flags are 0 for them.

That’s the problem. It is written in 0 then consumed in 2, but not used in subpass 1. Vulkan specification says that attachments are trashed unless they are listed in Preserved Attachments.

Shouldn’t depend on OS and environment much. Only caveat is wherefrom you consume SDK\layers. The updated distribution is at LunarXchange. Other meddling middlemen may supply only old versions.

We shall see. Untangling the sync is bit experimental, and bugs in the layers are also not uncommon.

I don’t have your code, but I can try Sasha’s stuff later. Occasionally I also contribute to Layers, so I should be able to try unpublished fixes as last resort before reporting a bug.

Similar problem seems somewhat reported at Possible false positive WAW for image layout transition during queue family transfer (QueueSubmit) · Issue #7024 · KhronosGroup/Vulkan-ValidationLayers · GitHub

Strategy to brute force finding what offends the layers could be to serialize everything and set it to eAllComands with eMemoryRead|eMemoryWrite, and then iteratively loosen the sync until it shows the error(s).

wusikijeronii · March 11, 2024, 1:26pm

Did I write it right?

    dependencies[0]
        .setSrcSubpass(vk::SubpassExternal)
        .setDstSubpass(0)
        .setSrcStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setDstStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput |
                         vk::PipelineStageFlagBits::eEarlyFragmentTests)
        .setSrcAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDstAccessMask(vk::AccessFlagBits::eMemoryWrite);

    dependencies[1]
        .setSrcSubpass(vk::SubpassExternal)
        .setDstSubpass(1)
        .setSrcStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setDstStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setSrcAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDstAccessMask(vk::AccessFlagBits::eMemoryWrite);

    dependencies[2]
        .setSrcSubpass(0)
        .setDstSubpass(2)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setDstStageMask(vk::PipelineStageFlagBits::eFragmentShader)
        .setSrcAccessMask(vk::AccessFlagBits::eMemoryWrite)
        .setDstAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

    dependencies[3]
        .setSrcSubpass(1)
        .setDstSubpass(2)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setDstStageMask(vk::PipelineStageFlagBits::eFragmentShader)
        .setSrcAccessMask(vk::AccessFlagBits::eMemoryWrite)
        .setDstAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

    dependencies[4]
        .setSrcSubpass(2)
        .setDstSubpass(vk::SubpassExternal)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setDstStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setSrcAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDstAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

I stil get errors.
I also added mainpassAttachment as input attachment in subpass 1.

wusikijeronii · March 11, 2024, 9:02pm

Today I launched my code on other machines. I tried a Linux server and another PC on Windows. But I can’t reproduce the issue. The more I try to learn more about this problem, the more I think that it is a compatibility issue. On the other PC, I also updated the Vulkan SDK to the same version, drivers, and OS, but I still don’t see any warnings. Guys, what do you think?

krOoze · March 12, 2024, 1:48am

Sasha Willems inputattachments example did show errors to me, but I managed to clear them all by fixing its sync bugs.

As previously said, I wouldn’t\shouldn’t expect this aspect of validation to be platform-dependent. If it shows nothing on other platforms, perhaps the layers (and the optional sync validation) were not enebled properly?

PS: small possibility is that some bad implicit layers trigger the error, and not you.

wusikijeronii · March 12, 2024, 8:36am

Ooops… I forgot to turn on synchronization in the validation area. I thought it would be set automatically when I switched the validation layer’s modes.
But I found the error in my code. I fixed it! Thanks a lot for the help and the explanation. A few days ago, it was something unknown to me. Now I understand how it works and what depends on what.

About the error:
I used the last code with five dependencies. In these dependencies, I set access flags to wait until input attachments are ready to read. But the final layouts in these attachments were set to eShaderReadOnlyOptimal instead of eColorAttachmentOptimal. Basically, input attachments work in the eColorAttachmentOptimal layout. So, here there was really missed logic. Because syncronization was set to wait for eColorAttachmentOptimal but the actual image was presented in eShaderReadOnlyOptimal, syncronization didn’t work properly.
My solution is to set the correct final layout in the vk::AttachmentDescription setup and mirrorly reflect the setted logic in dependencies.

wusikijeronii · March 13, 2024, 4:06pm

@krOoze On my laptop, it doesn’t work. Only on PC. So then I tried to simplify my rendering process. And now I just render empty images. I removed all additional subpasses. It works. Then I added one subpass. Yes, I received some errors, but I fixed them. Then I marked the subpass 0 attachment as an input attachment in subpass 1. But it doesn’t work. Also, tried to mark src/dst as eAllGraphics. But even in this case, I get errors.
Let’s start again with this code:

void Swapchain::createRenderPass()
{
    vk::AttachmentDescription mainpassAttachment;
    mainpassAttachment.setFormat(_imageFormat)
        .setSamples(vk::SampleCountFlagBits::e1)
        .setLoadOp(vk::AttachmentLoadOp::eClear)
        .setStoreOp(vk::AttachmentStoreOp::eDontCare)
        .setStencilLoadOp(vk::AttachmentLoadOp::eDontCare)
        .setStencilStoreOp(vk::AttachmentStoreOp::eDontCare)
        .setInitialLayout(vk::ImageLayout::eUndefined)
        .setFinalLayout(vk::ImageLayout::eColorAttachmentOptimal);
    vk::AttachmentReference mainpassAttachmentRef(0, vk::ImageLayout::eColorAttachmentOptimal);

    vk::AttachmentDescription colorAttachment;
    colorAttachment.setFormat(_imageFormat)
        .setSamples(vk::SampleCountFlagBits::e1)
        .setLoadOp(vk::AttachmentLoadOp::eClear)
        .setStoreOp(vk::AttachmentStoreOp::eStore)
        .setStencilLoadOp(vk::AttachmentLoadOp::eDontCare)
        .setStencilStoreOp(vk::AttachmentStoreOp::eDontCare)
        .setInitialLayout(vk::ImageLayout::eUndefined)
        .setFinalLayout(vk::ImageLayout::ePresentSrcKHR);
    vk::AttachmentReference colorAttachmentRef(1, vk::ImageLayout::eColorAttachmentOptimal);

    vk::AttachmentReference unusedAttachmentRef;
    unusedAttachmentRef.attachment = vk::AttachmentUnused;
    unusedAttachmentRef.layout = vk::ImageLayout::eUndefined;

    std::array<vk::AttachmentReference, 1> inputAttachments;
    inputAttachments[0] = vk::AttachmentReference(0, vk::ImageLayout::eShaderReadOnlyOptimal);

    std::array<vk::SubpassDescription, 2> subpasses;
    subpasses[0]
        .setPipelineBindPoint(vk::PipelineBindPoint::eGraphics)
        .setColorAttachmentCount(1)
        .setPColorAttachments(&mainpassAttachmentRef)
        .setPDepthStencilAttachment(&unusedAttachmentRef);

    subpasses[1]
        .setPipelineBindPoint(vk::PipelineBindPoint::eGraphics)
        .setColorAttachmentCount(1)
        .setPColorAttachments(&colorAttachmentRef)
        .setPDepthStencilAttachment(&unusedAttachmentRef)
        .setInputAttachmentCount(1)
        .setPInputAttachments(inputAttachments.data());

    std::array<vk::SubpassDependency, 3> dependencies;
    dependencies[0]
        .setSrcSubpass(vk::SubpassExternal)
        .setDstSubpass(0)
        .setSrcStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setDstStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setSrcAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDstAccessMask(vk::AccessFlagBits::eColorAttachmentWrite);

    dependencies[1]
        .setSrcSubpass(0)
        .setDstSubpass(1)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setDstStageMask(vk::PipelineStageFlagBits::eFragmentShader)
        .setSrcAccessMask(vk::AccessFlagBits::eColorAttachmentWrite)
        .setDstAccessMask(vk::AccessFlagBits::eInputAttachmentRead)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

    dependencies[2]
        .setSrcSubpass(1)
        .setDstSubpass(vk::SubpassExternal)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setDstStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setSrcAccessMask(vk::AccessFlagBits::eColorAttachmentRead | vk::AccessFlagBits::eColorAttachmentWrite)
        .setDstAccessMask(vk::AccessFlagBits::eMemoryRead)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

    std::array<vk::AttachmentDescription, 2> attachments = {mainpassAttachment, colorAttachment};
    vk::RenderPassCreateInfo renderPassInfo;
    renderPassInfo.setAttachmentCount(static_cast<u32>(attachments.size()))
        .setPAttachments(attachments.data())
        .setSubpassCount(2)
        .setPSubpasses(subpasses.data())
        .setDependencyCount(dependencies.size())
        .setPDependencies(dependencies.data());

    if (_device.vkDevice().createRenderPass(&renderPassInfo, nullptr, &_renderPass, _device.vkLoader()) !=
        vk::Result::eSuccess)
        throw std::runtime_error("Failed to create render pass");
}

Errors:

ERROR Validation Error: [ SYNC-HAZARD-WRITE-AFTER-WRITE ] Object 0: handle = 0xb097c90000000027, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0x5c0ec5d6 | vkCmdEndRenderPass(): Hazard WRITE_AFTER_WRITE in subpass 1 for attachment 0 color aspect during store with storeOp VK_ATTACHMENT_STORE_OP_DONT_CARE. Access info (usage: SYNC_COLOR_ATTACHMENT_OUTPUT_COLOR_ATTACHMENT_WRITE, prior_usage: SYNC_IMAGE_LAYOUT_TRANSITION, write_barriers: SYNC_FRAGMENT_SHADER_INPUT_ATTACHMENT_READ, command: vkCmdNextSubpass, seq_no: 2, subcmd: 1, renderpass: VkRenderPass 0xb097c90000000027, reset_no: 1)

ERROR Validation Error: [ SYNC-HAZARD-WRITE-AFTER-READ ] Object 0: handle = 0x2280bc04818, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x376bc9df | vkQueueSubmit(): Hazard WRITE_AFTER_READ for entry 0, VkCommandBuffer 0x2280be02750, Submitted access info (submitted_usage: SYNC_IMAGE_LAYOUT_TRANSITION, command: vkCmdNextSubpass, seq_no: 2, subcmd: 1, renderpass: VkRenderPass 0xb097c90000000027, reset_no: 1). Access info (prior_usage: SYNC_PRESENT_ENGINE_SYNCVAL_PRESENT_ACQUIRE_READ_SYNCVAL, read_barriers: VK_PIPELINE_STAGE_2_FRAGMENT_SHADER_BIT|VK_PIPELINE_STAGE_2_LATE_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_2_COLOR_ATTACHMENT_OUTPUT_BIT|VK_PIPELINE_STAGE_2_BOTTOM_OF_PIPE_BIT, , batch_tag: 96, vkAcquireNextImageKHR aquire_tag:96: VkSwapchainKHR 0x9fde6b0000000014, image_index: 0image: VkImage 0xdd3a8a0000000015).

About SYNC_PRESENT_ENGINE_SYNCVAL_PRESENT_ACQUIRE_READ_SYNCVAL. You asked before about this. Yes, I wait for a semaphore. In the wait stages, I set eColorAttachmentOutput.

Can you explain to me what I did wrong? How can I fix sync warnings?

UPD: Using your approach, it seems I found the wrong dependency.

    dependencies[1]
        .setSrcSubpass(0)
        .setDstSubpass(1)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setDstStageMask(vk::PipelineStageFlagBits::eAllGraphics)
        .setSrcAccessMask(vk::AccessFlagBits::eColorAttachmentWrite)
        .setDstAccessMask(vk::AccessFlagBits::eMemoryRead | vk::AccessFlagBits::eMemoryWrite)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

In this case, the error has disappeared. But how can I find the correct flags?

krOoze · March 13, 2024, 5:48pm

As I said, I wouldn’t expect differences here between PCs. This is not platform dependent code, really. Make sure your laptop also has updated layers (1.3.275).

wusikijeronii · March 13, 2024, 6:06pm

I’ve already updated the validation layers on the laptop. Let’s ignore this fact at this moment. On my laptop, I found the barrier of unsync operation (dependency[1]). Let’s think about how I can set proper flags to avoid the issue.

krOoze · March 13, 2024, 9:41pm

Ignoring facts is good way to waste time chasing rabbits while debugging.

These sync demands are the same regardless a platform. If these reported errors differ anyway, it is a significant fact, and an indication that we are missing something.

I don’t immediately see anything offensive in the parts of the code you have shown. The error with vkAcquire is partially outside the scope of the code you have shown.

The first error text is somewhat nonsensical. Store op is never synchronized against prior use. That is always automatic.

Anyway here’s the fixed Sasha Willems example if you want to try it: GitHub - krOoze/WillemsVulkan at fix_inputattachments_sync

wusikijeronii · March 13, 2024, 10:09pm

On the PC, I was able to reproduce the validation layer errors. It was just necessary to check all the boxes in the Vulkan Configurator. On a laptop, it is enough to set the synchronization mode in the validation preset.

Your example works without any errors for me. Thank you for sharing. So, new day, new tries. About vkAcquire, I don’t think it is the reason for the issue. But my code is here:


    vk::Result acquireNextImage(u32 *imageIndex)
    {
        return _device.vkDevice().acquireNextImageKHR(_swapchain, UINT64_MAX, _imageAvailableSemaphores[_currentFrame],
                                                      nullptr, imageIndex, _device.vkLoader());
    }

It seems I have to make some tests according to your example until the issue appears (due to my code).

wusikijeronii · March 16, 2024, 7:20pm

@krOoze It seems I found the issue’s reason. First, I just copied code from the render pass creation function in your example, but I still got the error. Next, I cloned the VulkanTutorial project and also copied your code. Everthing worked. Then I started to include code from my project in my clone. Code by code. And finally, I found the reason for the issue. The problem appears when I use secondary command buffers. May I have missed something?
My changes to work with secondary buffers:

For every subpass, I created command buffers with the secondary command buffer level flag, with the count equal to the primary command buffers.
vkCmdBeginRenderPass I have to set SubpassContents as SecondaryCommandBuffers instead of Inline.
In every subpass rendering block, I receive the secondary buffer by current frame index (same as without secondary command buffers).
I should begin and end using a secondary command buffer. And in the end, I have to call vkCmdExecuteCommands.
Finally, in vkCmdNextSubpass, I have to change Subpass Contents to VK_SUBPASS_CONTENTS_SECONDARY_COMMAND_BUFFERS.

Do I miss something? How can I syncronize secondary command buffers to avoid this warning?

Also I attach the draw frame function code:

void App::drawFrame()
{
    auto commandBuffer = renderer.beginFrame();
    updateUniformBuffer(renderer._currentFrameIndex);
    renderer.beginRenderPass(commandBuffer);
    auto secondary0 = _commandBuffers0[renderer._currentFrameIndex];
    {
        vk::CommandBufferInheritanceInfo inheritanceInfo;
        inheritanceInfo.setRenderPass(renderer._swapchain->renderPass()).setSubpass(0);
        vk::CommandBufferBeginInfo beginInfo;
        beginInfo
            .setFlags(vk::CommandBufferUsageFlagBits::eRenderPassContinue |
                      vk::CommandBufferUsageFlagBits::eOneTimeSubmit)
            .setPInheritanceInfo(&inheritanceInfo);
        secondary0.begin(beginInfo, device.vkLoader);
        secondary0.end(device.vkLoader);
    }
    commandBuffer.executeCommands(1, &secondary0, device.vkLoader);
    vkCmdNextSubpass(commandBuffer, VK_SUBPASS_CONTENTS_SECONDARY_COMMAND_BUFFERS);
    auto secondary1 = _commandBuffers1[renderer._currentFrameIndex];
    {
        vk::CommandBufferInheritanceInfo inheritanceInfo;
        inheritanceInfo.setRenderPass(renderer._swapchain->renderPass()).setSubpass(1);
        vk::CommandBufferBeginInfo beginInfo;
        beginInfo
            .setFlags(vk::CommandBufferUsageFlagBits::eRenderPassContinue |
                      vk::CommandBufferUsageFlagBits::eOneTimeSubmit)
            .setPInheritanceInfo(&inheritanceInfo);
        secondary1.begin(beginInfo, device.vkLoader);
    }
    recordCommandBuffer(secondary1, renderer._currentImageIndex);
    secondary1.end(device.vkLoader);
    commandBuffer.executeCommands(1, &secondary1, device.vkLoader);
    commandBuffer.endRenderPass(device.vkLoader);
    renderer.endFrame(commandBuffer);
}

And renderer commands:

vk::CommandBuffer Renderer::beginFrame()
{
    _swapchain->waitForFences();
    _swapchain->acquireNextImage(&_currentImageIndex);
    _isFrameStarted = true;
    vk::CommandBuffer commandBuffer = _commandBuffers[_currentFrameIndex];
    vk::CommandBufferBeginInfo beginInfo;
    if (commandBuffer.begin(&beginInfo, _device.vkLoader) != vk::Result::eSuccess)
        throw std::runtime_error("Failed to begin recording command buffer");
    return commandBuffer;
}

void Renderer::beginRenderPass(vk::CommandBuffer commandBuffer)
{
    vk::RenderPassBeginInfo renderPassInfo;
    auto clearValues = _swapchain->clearValues();
    renderPassInfo.setRenderPass(_swapchain->renderPass())
        .setFramebuffer(_swapchain->vkFramebuffer(_currentImageIndex))
        .setRenderArea({{0, 0}, _swapchain->extent()})
        .setClearValueCount(static_cast<u32>(clearValues.size()))
        .setPClearValues(clearValues.data());
    commandBuffer.beginRenderPass(&renderPassInfo, vk::SubpassContents::eSecondaryCommandBuffers, _device.vkLoader);
}

void Renderer::endFrame(vk::CommandBuffer commandBuffer)
{
    commandBuffer.end(_device.vkLoader);
    _swapchain->submitCommandBuffers(&commandBuffer, &_currentImageIndex);
    _isFrameStarted = false;
    _currentFrameIndex = (_currentFrameIndex + 1) % Swapchain::maxFramesPerTime;
}

Current error:

ERROR Validation Error: [ SYNC-HAZARD-WRITE-AFTER-WRITE ] Object 0: handle = 0xd5b26f0000000010, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0x5c0ec5d6 | vkCmdEndRenderPass(): Hazard WRITE_AFTER_WRITE in subpass 1 for attachment 1 color aspect during store with storeOp VK_ATTACHMENT_STORE_OP_DONT_CARE. Access info (usage: SYNC_COLOR_ATTACHMENT_OUTPUT_COLOR_ATTACHMENT_WRITE, prior_usage: SYNC_IMAGE_LAYOUT_TRANSITION, write_barriers: SYNC_FRAGMENT_SHADER_INPUT_ATTACHMENT_READ, command: vkCmdNextSubpass, seq_no: 3, subcmd: 1, renderpass: VkRenderPass 0xd5b26f0000000010, reset_no: 1)

I used your code to create a render pass. And here, too, if I set eGraphics and eMemoryRead / eMemoryWrite in dependencies[2], then the error disappears. Apparently some additional logic is needed for secondary buffers, but I don’t understand which one.
UPD: Seems, I’ve found need flags:

    dependencies[0]
        .setSrcSubpass(vk::SubpassExternal)
        .setDstSubpass(0)
        .setSrcStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setDstStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput |
                         vk::PipelineStageFlagBits::eEarlyFragmentTests)
        .setSrcAccessMask(vk::AccessFlagBits::eNone)
        .setDstAccessMask(vk::AccessFlagBits::eColorAttachmentWrite | vk::AccessFlagBits::eDepthStencilAttachmentWrite)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);
    dependencies[1]
        .setSrcSubpass(vk::SubpassExternal)
        .setDstSubpass(1)
        .setSrcStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setDstStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setSrcAccessMask(vk::AccessFlagBits::eNone)
        .setDstAccessMask(vk::AccessFlagBits::eColorAttachmentWrite)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

    // This dependency transitions the input attachment from color attachment to
    // shader read
    dependencies[2]
        .setSrcSubpass(0)
        .setDstSubpass(1)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput |
                         vk::PipelineStageFlagBits::eEarlyFragmentTests | vk::PipelineStageFlagBits::eLateFragmentTests)
        .setDstStageMask(vk::PipelineStageFlagBits::eFragmentShader | vk::PipelineStageFlagBits::eLateFragmentTests |
                         vk::PipelineStageFlagBits::eColorAttachmentOutput)
        .setSrcAccessMask(vk::AccessFlagBits::eColorAttachmentWrite | vk::AccessFlagBits::eDepthStencilAttachmentRead |
                          vk::AccessFlagBits::eDepthStencilAttachmentWrite)
        .setDstAccessMask(vk::AccessFlagBits::eInputAttachmentRead | vk::AccessFlagBits::eColorAttachmentWrite |
                          vk::AccessFlagBits::eDepthStencilAttachmentWrite)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);
    dependencies[3]
        .setSrcSubpass(1)
        .setDstSubpass(vk::SubpassExternal)
        .setSrcStageMask(vk::PipelineStageFlagBits::eColorAttachmentOutput |
                         vk::PipelineStageFlagBits::eLateFragmentTests)
        .setDstStageMask(vk::PipelineStageFlagBits::eBottomOfPipe)
        .setSrcAccessMask(vk::AccessFlagBits::eColorAttachmentWrite | vk::AccessFlagBits::eDepthStencilAttachmentWrite)
        .setDstAccessMask(vk::AccessFlagBits::eNone)
        .setDependencyFlags(vk::DependencyFlagBits::eByRegion);

@krOoze Can I ask your opinion? Is it correct? Why didn’t your flags work using secondary command buffers?

krOoze · March 18, 2024, 3:50am

Secondary command buffers should just be semantically identical to inline recordings.

It is empty? Should have a draw…

Heavily abstracted to be able to see anything…

As previously said, it is somewhat nonsensical that Store Op could hazard with anything prior to it. That is automatic sync. I would report it as bug on that alone.

Assuming it is not fixed already. Since last distribution, there seem to have been like 10 sync validation issues closed already.

Hm. You could try the Store Op flags specifically (COLOR+ATTACHMENT_WRITE). Which I guess you already did.

I guess the layers indeed have problem decoding the situation.

krOoze · March 21, 2024, 8:39pm

Any change with the newly released SDK (v280)?

wusikijeronii · March 22, 2024, 5:17am

Same behavior. I updated the Vulkan SDK and loaded the test project with my last flags. No errors.
Then, I reverted flags to flags from your example. Same error.
I change secondary command buffers to primary ones. And everything works again.
My project has an early and early development state, and I just thought sooner or later anyone would fix it. But now I think that if nobody says anything about the issue, no one will fix it. So, I’ll create a ticket on GitHub. Later. I need to prepare an easily reproducible example.
UPD: Added the issue.