Secondary Command Buffers

I am finding that secondary command buffers are rather restrictive. What I’d really like to see is the ability for pre-recorded secondary commandbuffers to inherit push constants and push descriptors from the corresponding bind point.

Is there any likelihood that this sort of thing may come in down the line ?

I’d guess no.

The main purpose of secondary CBs is to allow multiple threads to generate rendering commands, while still allowing the render pass model to work and be effective. That’s why render pass state is the only thing (besides counters) that gets inherited by secondary CBs.

Presumably, what you want is to have a static secondary CB for each mesh group you render, and then just throw it on the pile with a few commands between them to change where the next static CB gets its data from.

This wouldn’t work because Vulkan doesn’t allow more than two levels of CBs (primary and secondary). Which means that you would have to be issuing these push constants/descriptors in the primary CB. Which means you cannot thread building the CB at all. You can’t have one thread working on some of the commands and another thread working on others, with them all going into the same subpass of the primary CB.

To make what you want work would require a whole new kind of CB: a tertiary CB (analogous to D3D12’s bundles).

Really, just issue the rendering commands manually; it’s probably not going to be that much more expensive on the CPU or the GPU. Alternatively, you can put the data into buffers and issue rendering commands indirectly, to minimize CPU cache thrashing.

:slight_smile: My guess was no also.

However, for my use cases at least, I think the two command buffer levels would be sufficient. I’m sort of thinking about generated secondary command buffers that stay alive across several frames. That is fine as is, but you hit problems with resources that are per frame. Thinks like a ping-pong buffer for ssao or skybox cubemap.

So my nirvana would look like:

  • begin primary cb
  • push descriptor with ssao & skybox binding
  • compute shaders etc that generate ssao & skybox
  • begin renderpass
  • execute secondary cb
  • endpass
  • etc, submit, rinse and repeat.

Anyway, just wishful thinking, many other ways to skin the cat.


PS. interesting titbit is that the nvidia drivers already work this way, ignoring validation problems :slight_smile:

Thinks like a ping-pong buffer for ssao or skybox cubemap.

The thing I don’t understand is this: why does it matter? How many commands are you really sending to do ping-pong rendering or to draw a skybox? A skybox draw is binding descriptor sets, some vertex buffers, a pipeline, maybe some push constants, and then a rendering command. That’s 4-5 commands; do you really expect to see a significant performance gain from packing them into a CB?

Indeed, for so few drawing commands, using a secondary CB might actually be more expensive than just writing them. Why? Because a lot of the aspects of the rendering state are hard-coded. You don’t have to read the number of vertices to send in your vkCmdDraw command; it’s hard-coded into your application. By contrast, invoking a secondary CB would mean doing a copy of all of the data in the CB, which could cause more data reads (and thus potentially more cache thrashing) than invoking the commands directly.

Also, the skybox could be in a static secondary CB. But not ping-ponging, due to having to deal with render pass attachments.

Sorry for being unclear.

It wouldn’t be the ssao or skybox that is using the secondary cb, it is all the rest of the geometry that then does lookups into the ssao or skybox (for environmental lighting etc) or for many other reasons or cases.

And the ping-pong is a typical example of things that change every frame, like ssao temporal anti aliasing or dynamic skybox time of day.

Please, I’m not trying to argue whether to use secondary command buffers. I’m suggesting that if you want to use them, then per frame bindings would be nice.