Why on earth does this renderpass setup NOT raise a synchronization validation error?

[johannesugb]

I wanted to provoke a synchronization validation error by setting all the external subpass dependencies of stages and memory accesses to NONE, but I do not get synchronization validation errors. Why not?

Here’s my setup as captured with RenderDoc:

vkCmdBeginRenderPass(C=Clear, D=Clear) // using RP#1
vkCmdDrawIndexed(....)  // several of these
vkCmdEndRenderPass(C=Store, D=Store)
vkCmdBeginRenderPass(C=Load, D=Load) // using RP#2
vkCmdDrawIndexed(...)
vkCmdEndRenderPass(C=Store, D=Store)

Both render passes use the same color C and depth D attachment images (i.e., 2 attachments in total). Each render pass has one single subpass.

Subpass dependencies of RP#1 are as follows:

dependencyCount   2
	pDependencies     VkSubpassDependency2[2]
		[0]             VkSubpassDependency2()
			sType         VK_STRUCTURE_TYPE_SUBPASS_DEPENDENCY_2
			pNext         NULL
			srcSubpass    UINT32_MAX
			dstSubpass    0
			srcStageMask  VK_PIPELINE_STAGE_NONE
			dstStageMask  VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT | VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT | VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT
			srcAccessMask VK_ACCESS_NONE
			dstAccessMask VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT
			dependencyFlags VkDependencyFlagBits(0)
			viewOffset    0
		[1]             VkSubpassDependency2()
			sType         VK_STRUCTURE_TYPE_SUBPASS_DEPENDENCY_2
			pNext         NULL
			srcSubpass    0
			dstSubpass    UINT32_MAX
			srcStageMask  VK_PIPELINE_STAGE_NONE
			dstStageMask  VK_PIPELINE_STAGE_NONE
			srcAccessMask VK_ACCESS_NONE
			dstAccessMask VK_ACCESS_NONE
			dependencyFlags VkDependencyFlagBits(0)
			viewOffset    0

Its C and D layouts are specified as follows (initial → subpass 0 → final), respectively:

VK_IMAGE_LAYOUT_UNDEFINED -> VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL -> VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL
VK_IMAGE_LAYOUT_UNDEFINED -> VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL -> VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL

Subpass dependencies of RP#2 are as follows:

dependencyCount   2
	pDependencies     VkSubpassDependency2[2]
		[0]             VkSubpassDependency2()
			sType         VK_STRUCTURE_TYPE_SUBPASS_DEPENDENCY_2
			pNext         NULL
			srcSubpass    UINT32_MAX
			dstSubpass    0
			srcStageMask  VK_PIPELINE_STAGE_NONE
			dstStageMask  VK_PIPELINE_STAGE_NONE
			srcAccessMask VK_ACCESS_NONE
			dstAccessMask VK_ACCESS_NONE
			dependencyFlags VkDependencyFlagBits(0)
			viewOffset    0
		[1]             VkSubpassDependency2()
			sType         VK_STRUCTURE_TYPE_SUBPASS_DEPENDENCY_2
			pNext         NULL
			srcSubpass    0
			dstSubpass    UINT32_MAX
			srcStageMask  VK_PIPELINE_STAGE_NONE
			dstStageMask  VK_PIPELINE_STAGE_NONE
			srcAccessMask VK_ACCESS_NONE
			dstAccessMask VK_ACCESS_NONE
			dependencyFlags VkDependencyFlagBits(0)
			viewOffset    0

Its C and D layouts are specified as follows (initial → subpass 0 → final), respectively:

VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL -> VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL -> VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL
VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL -> VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL -> VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL

---

Note:
I know that synchronization validation is working because if I change the first subpass dependency to all VK_PIPELINE_STAGE_NONE and VK_ACCESS_NONE, then I get the following validation error due to the layout transition hazard:

Validation Error: [ SYNC-HAZARD-WRITE_AFTER_WRITE ] Object 0: handle = 0x12fb2600000002c0, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0xfdf9f5e1 | vkCmdBeginRenderPass: Hazard WRITE_AFTER_WRITE vs. layout transition in subpass 0 for attachment 1 aspect depth during load with loadOp VK_ATTACHMENT_LOAD_OP_CLEAR.

---

Question:
But if I leave the subpass dependencies like stated above, I would definitely also expect a SYNC-HAZARD-WRITE_AFTER_WRITE because RP#2 writes into the same attachments as RP#1.

The layout stays the same, so no layout transition has to be performed, but I would say that there would still have to be a synchronization validation error complaining about the fact that RP#2 does not wait for RP#1 to finish its depth and color writes. Why doesn’t it complain?

RP#2 races RP#1, doesn’t it? Do I overlook something here?

[krOoze]

Why on earth won’t you ask at https://github.com/KhronosGroup/Vulkan-ValidationLayers?

[johannesugb]

I want to make sure that I do not overlook anything.
It should issue a warning/error, shouldn’t it?

If you agree, I’ll go ahead and post at GitHub - KhronosGroup/Vulkan-ValidationLayers: Vulkan Validation Layers. ^^

[krOoze]

Hm, let’s see.

The weird implicit layout sync comes to mind. But you say there is no layout change.

Otherwise it does feels like synchronization hazard between Store Op of RP1 and Load Op of RP2.

Load\Store Ops should only happen if something actually “uses” the attachment, which can be bit hairy to determine. Does your shaders actually write something, and is the rasterization enabled, and write masks good, etc? This could perhaps require more brutal validation. Do you have the Shader Validation feature also enabled?

Probably the validation will just be unimplemented\incomplete. That’s normal. I mean sync validation itself was only relatively recently introduced…

[johannesugb]

Thanks for the pointers! I am pretty sure that there is no weird implicit layout sync in place, but it was an immensely interesting read. Thanks for pointing it out!

Double-checked everything again:

• both render passes write depth and colors,
• rasterization is enabled,
• write masks must be good because color/depth results are written indeed,
• enabled VK_VALIDATION_FEATURE_ENABLE_GPU_ASSISTED_EXT ,
• did not use VK_VALIDATION_FEATURE_DISABLE_SHADERS_EXT (i.e. the shader validation feature should be enabled ← I assume this is what you meant?!).
• Tested everything by manually setting up validation layers and also through Vulkan Configurator.

=> Nothing.

Yeah, I’d bet on synchronization validation being incomplete for this case. It just didn’t appear to me as natural—especially for my use case above. I believe that I’ve read that especially render pass sync was one of the priorities of validation synchronization and after establishing a total non-sync scenario, I definitely expected a synchronization validation error in this case.