How to merge results from multiple pipelines?

Vulkan
1

Greetings.

I’ve followed the most popular vulkan tutorial , finished it, and am currently learning to do less cargo cult and more understanding of what I am doing. Building off the tutorial, I am currently trying to merge the rendering results of three rendering pipelines into a single image.

What I expect are fragments generated by each pipeline to be depth-tested against each other and presented to the screen. No alpha blending or anything.

What I do:

  1. Get an image from the swap chain
  2. Get a free command buffer for each pipeline from the same pool.
  3. Record, single threadedly, commands for each buffer.
  4. Get all the recorded buffers, submit them to the queue.
  5. Present results.

If I do this for each pipeline separately, it works. Although I need to set the loadOp in color attachment for this pipeline to VK_ATTACHMENT_LOAD_OP_CLEAR, otherwise I get a hall of mirrors effect. Once set, I get no z-fighting, nice image, everything seems fine.

If I try to get the merged results… things happen.

If I omit VK_ATTACHMENT_LOAD_OP_CLEAR for every pipeline, all things are drawn, but a massive hall of mirrors effect ensues. Seems like depth testing between the pipelines is not done.

If I enable “blending” of results, hall of mirrors is gone, but, obviously, all fragments blend on each other. There is also a lot of flickering.

Combinations of all of these don’t solve my issue.

I’ve been at it for the last three days. Reading tutorials, articles and specification. I don’t have a faintest idea what to do next.

My questions, if anyone would like to throw some aid

  1. If an attachment has its load/store operations set to anything substantial - does this happen per fragment or image? For example - If I make 100x100 render window and produce one pixel, with the rest untouched - will the attachment load just for that one fragment? This seems to happen per image, but I might be missing something.

  2. What happens in my example looks like some sort of a race condition between each pipeline as they read/write to their color/depth/resolve attachments. All of them have the same three images bound - respectively.

Does it mean I need to add a barrier/fence somewhere to synchronize between each pipeline?

Pipeline barriers seem to be per command buffer. I didn’t find any vkCmdFence or vkCmdSemaphore to share a semaphore between several pipelines. Should I just submit a single command buffer? I tried different combinations of buffers / submits but to no avail. So I guess - how to synchronize between several pipelines if they were submitted all at once?

  1. Alternatively - should I just bind different images to attachments of each pipeline and then ‘manually’ merge their results? This seems weird, but considering how expressive Vulkan is, I would not surprise me.

On unrelated note.

vkQueueSubmit, well, submits commands to the GPU driver and they can be asynchronously executed from that point - right/wrong?

vkQueuePresentKHR waits (?) until those commands are executed. Gets the image to the screen and signals the attached semaphore? Or does it just assume that we are done with executing commands and are going to do other things?

I guess - what I am asking is if I do a tight loop like this:

for( int x = 0; x < 10; ++x )
{
   submit;
   present;
}

Will I get x frames rendered, or will only from the last ‘present’ - or is it an undefined behavior of some sort to call for a present without waiting for the semaphores passed to vkQueuePresentKHR?

Anyhow - I appreciate any help. I knew that Vulkan is vast. But I’ve been at it for the last two months and it’s been a humbling experience. I’ve been code monkeying for money for the last 18 years…

2

I find the description of your code to be confusing.

Pipelines do not have a loadOp. Render passes do however. But since all three pipelines are rendering to (presumably?) the same image, they should all be part of the same render pass.

So why are you making these into separate render passes?

What I expect are fragments generated by each pipeline to be depth-tested against each other and presented to the screen.

What do you mean by “each other”? Depth testing is an ordered operation; you test against the depth values written previously, not against depth values from drawing commands that haven’t been processed yet.

3

I am no graphics whiz, but I will share what I do see.

You are absolutely correct in supposing you have a ‘race-condition’. You are effectively asking three entirely separate entities to write to a single area of memory simultaneously, and the outcome is necessarily non-deterministic, or at least not what you intended. The different results you describe under your different scenarios are the natural consequences, to be anticipated.

What I expect are fragments generated by each pipeline to be depth-tested against each other and presented to the screen. No alpha blending or anything.

I am not sure what leads you to this expectation, as you later correctly point out that: considering how expressive Vulkan is, I would not surprise me. (spelling copied verbatim) I expect what you mean is that Vulkan is explicit, and that it certainly is, by design.

So, you cannot expect anything to happen, that you did not specifically make happen yourself. In that same vein, it is not clear to me what you actually mean by merging the results. Essentially, you are producing three unrelated images, but writing them into a common buffer. Unless you are going to do something to effect this blending or merging, it cannot ever happen. And yes, if you simply blend (as with alpha) the ‘colours’, you will simply have some arbitrary combination of your three images.

I presume that you are simply doing this to experiment, as I cannot see any actual use for the path you are choosing to take – if that is incorrect, do let us know. On that basis, you have a multitude of different ways to resolve the ‘issue’.

If you really do intend to combine the output from three different pipelines, you will need to synchronize that output. BUT, without understanding what it is you are actually trying to achieve, it is challenging to offer any useful hints.

If you care to clarify, I am sure folks will be eager to assist you.

As to your second question. May I suggest that you post it separately? That way, future users will be able to find the question and make use of the answers. If anyone just answers that here, it will simply be lost…

4

Sorry for being confusing.

  1. Pipelines have passes - mine has 1
  2. Passes have attachments - mine has 3
  3. Attachments have a loadOp.

Depth testing is an ordered operation; you test against the depth values written previously, not against depth values from drawing commands that haven’t been processed yet.

I want to draw three things:

  1. A quad parallel to XY plane on Z=0.99 to fill the entire screen with something. This uses one pipeline and its own vertex/fragment shader. I do this to fill the entire screen with something and to have a backdrop against which my geometry is going to be shown. This part will create a sky box at some point.

  2. A grid of lines which, essentially, are a world coordinate system axes. This uses another pipeline, with lines instead of triangles as primitives.

  3. A mesh in yet another pipeline, with its own vertex/fragment shader. Once I figure instancing, I’m going to show more meshes / instances.

Since each of these uses a different rendering method, I’m using a different rendering pipeline for each. They all write to the same color/depth/aa resolve buffers.

I’ll try to be concise here - but I’m not a native English speaker and, as I said before, I’m cargo culting so far - so I might not use the correct nomenclature.

My problem(s):

A pass attachment has a ‘loadOp’. I reckon that in 2nd and subsequent subpasess, this field specifies what to do with the value produced by the previous subpass. But for the first subpass - if I set it to VK_ATTACHMENT_LOAD_OP_CLEAR - then the entire image gets cleared, even if the rasterizer would never produce a fragment in that region.

I presupposed that reads/writes to a depth buffer are, for the lack of better term, atomic. If pipeline_1 produced a fragment at (.3,.3) and depth of 0.5, and pipeline_2 produced a fragment at (.3,.3) and depth of 0.3 then regardless of ordering of these operations, I should get the same result because:

  • pipeline_1 would discard its fragment if pipeline_2 already wrote its value
  • pipeline_2 would would overdraw whatever pipeline_1 already wrote to the color buffer

I get the same flickering results regardless if I record all commands, in order, to a single command buffer and submit that, or do three separate command buffers and submit them at once.

5

Yes. I figured as much. Now the issue is: how to resolve this. I’m, obviously, doing something wrong or incompletely. Solely because of my incompetence and lack of understanding of what am I doing.

I described what I intend to achieve in response to Alfonse_Reinheart. In short - three rendering operations using different rendering methods which I, wrongly, assumed would order themselves via writes to the depth buffer. This is what mathematics tells me ought to happen, this is what was happening when I was doing it in a different context, but what I was doing previously was not that close to the metal.

I can only describe what I want in terms of a result, not methods. For now:

  • render a quad that fills the entire screen and creates as ‘sky box’
  • render axes of the world coordinate system
  • render an example mesh
    Each operation uses a different vertex/fragment shader and, at some point, there are going to be more meshes using more shaders.
6

That helps a lot. Don’t be too hard on yourself; we ask questions because we want to learn. So that we can become more competent.

Where would you like to see the coordinate axes, in relation to the skybox and mesh?

7

Since each of these uses a different rendering method, I’m using a different rendering pipeline for each.

But all of those rendering pipelines should be within the same subpass of the same render pass instance. I see no reason for any of them to be in a different render pass.

They all write to the same color/depth/aa resolve buffers.

In the same render pass instance. Therefore, they are using the same attachments, and that render pass uses the same loadOp.

A pass attachment has a ‘loadOp’. I reckon that in 2nd and subsequent subpasess, this field specifies what to do with the value produced by the previous subpass.

A render pass has a loadOp. The subpasses within a render pass do not.

I presupposed that reads/writes to a depth buffer are, for the lack of better term, atomic. If pipeline_1 produced a fragment at (.3,.3) and depth of 0.5, and pipeline_2 produced a fragment at (.3,.3) and depth of 0.3 then regardless of ordering of these operations

Within a subpass of a render pass instance, all drawing commands will execute such operations in submission order.

Note that I did not mention “pipelines” in that because pipelines don’t really matter. What matters is drawing commands; drawing commands are what executes pipelines. So long as all of your drawing commands are issued in order, in the same subpass, you’re fine.

The second you break subpasses, you lose this implicit synchronization. You would then need explicit synchronization. But there’s no need to break subpasses here; all of your drawing commands should happen within the same subpass of the same render pass instance.

So:

I get the same flickering results regardless if I record all commands, in order, to a single command buffer and submit that, or do three separate command buffers and submit them at once.

If they’re all in the same render pass instance (as they should be), it doesn’t matter how many (secondary) command buffers you use. But by the fact fact that you don’t call them secondary command buffers, I suspect that you are beginning a render pass instance for each of these pipelines, regardless of how many command buffers you use.

That’s bad. Don’t do that.

8

The axes are drawn in the world coordinate space.

9

I don’t follow. Here are some questions. I’m using Vulkan 1.0, if that makes a difference.

When creating a pipeline via vkCreateGraphicsPipelines I set, among other things ) the following VkGraphicsPipelineCreateInfo:

stageCount / pStages - pointers to shader modules used by this pipeline. When I set more than two ( vertex/fragment ) I get validation layer complaints.

pInputAssemblyState - among other things, specification of what kind of things I’m going to draw. How do I differentiate between lines and triangles in the pass of this pipeline?

renderPass - gets a handle to the render pass I want to use. There is no interface to make several passes, at least not that I found reading the docs.

Recording commands for each pipeline in order to a single buffer also does not solve the issue - even though the commands are in order. As a test I tried to relate each pipeline to the single instance of render pass. No success either.

So:

I can’t relate to this advice at all.

EDIT:
Unless you suggest that I should create one VkRenderPass, share it among all my VkPipelines and do my drawing in this manner:

vkCmdBeginRenderPass( // my pass );
vkBindPipeline( //pipeline_1 )
// do drawing off pipeline 1
vkBindPipeline( //pipeline_2 )
// do drawing off pipeline 2
vkBindPipeline( //pipeline_3 )
// do drawing off pipeline 3
...
vkBindPipeline( //pipeline_n )
// do drawing off n-th pipeline
vkCmdEndRenderPass();
10

I did not say “instance of a render pass;” I said “render pass instance”. It’s a term of art; you’ll find that specific phrase in the Vulkan glossary.

A VkRenderPass is like a class. When you begin a render pass, that is what creates a “render pass instance”. Everything that happens between that begin and end is within the same render pass instance. All of your rendering should happen within the same render pass instance.

A pipeline is created against a particular render pass (not instance). You can bind that pipeline within any render pass instance that uses that render pass (or a compatible one, but the rules of compatibility are basically that it’s an object created with the exact same parameters). A drawing command happens within a particular render pass instance.

As such:

Yes, that is exactly the thing I’m saying you ought to be doing.

1 Like