VkCommandBuffer is in use (pending state)

Vulkan
1

I am rendering different objects using Vulkan. During rendering, I need that some objects appear and others dissappear. Therefore, I have to recreate the command buffer each frame. During execution, if I resize the window, I get the validation error below (although sometimes I get no errors). However, this doesn’t happen if I don’t recreate the command buffer each frame (which will force to use always the same one).

Validation layer: Validation Error: [ VUID-vkFreeCommandBuffers-pCommandBuffers-00047 ] Object 0: handle = 0x1da13a3c4e0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x1ab902fc | Attempt to free VkCommandBuffer 0x1da13a3c4e0[] which is in use. The Vulkan spec states: All elements of pCommandBuffers must not be in the pending state (Vulkan® 1.3.211 - A Specification (with all registered extensions))

There is a lot of code, so I present below the general process performed for each frame:

  1. Update UBOs
  2. Create command buffer:
    2.1. vkFreeCommandBuffers() <<<
    2.2. vkAllocateCommandBuffers()
    2.3. vkBeginCommandBuffer()
    2.4. Record commands
    2.5. vkEndCommandBuffer()
  3. vkQueueSubmit <<<
  4. Handle window resizing (recreate Swap Chain), if applicable

If the window is resized, the swap chain is recreated (point 4). Let’s see in more detail this process:

  1. vkDeviceWaitIdle(device)
  2. vkFreeCommandBuffers() <<<
  3. For each object: vkDestroyPipeline(), vkDestroyPipelineLayout(), destroyUniformBuffers(), vkDestroyDescriptorPool()
  4. Destroy depth image, vkDestroyFramebuffer(), vkDestroyRenderPass(), Destroy swap chain images, and vkDestroySwapchainKHR()
  5. Create swap chain, render pass, depth resources, and framebuffer
  6. For each object: create graphics pipeline, UBO, descriptor pool and descriptor set.
  7. Create command buffer (same process we saw previously)

Now, in case it’s relevant, let’s see in more detail the step “create depth resources” (point 5):

  1. vkCreateImage()
  2. vkCreateImageView()
  3. Transition the layout of the image to a depth attachment:
    3.1. vkAllocateCommandBuffers()
    3.2. vkBeginCommandBuffer()
    3.3. vkCmdPipelineBarrier()
    3.4. vkEndCommandBuffer()
    3.5. vkQueueSubmit() <<<
    3.6. vkQueueWaitIdle()
    3.7. vkFreeCommandBuffers() <<<

After printing some info in the command line, I get that, after resizing the window, things happen in the following way (although sometimes no error is output):

Frame n: Window handling happens
Frame n+1: No problem arises here
Frame n+2: Validation error appears when execution reaches the “create command buffer” step (more specifically, the vkFreeCommandBuffers() call).

According to the Vulkan specification (command buffer lifecycle), queue submission changes command buffer state to pending, until the command buffer is executed. I have checked and re-checked, but cannot see why this validation error appears. Maybe I am missing something. Can you see why this error could happen?

2

Problem with pseudocode is it shows intentions, not bugs. Proper debugging practice when really stuck is to form a minimal example (which means the least amount of work to trigger the problem).

As a substitute debugging practice, I would suggest you give the command buffers labels, and use VK_LAYER_LUNARG_api_dump to show stream of events as they really happen on that command buffer.

Additionally you should have some synchronization proofs associated with stuff. You should be able to tell me why do you believe the command buffer is ready to be reused at the point it is used (which should include use of fences and stuff). By the looks of it, you use command buffers from frame n in frame n+2 but failed to convince the validation layers that you separated those uses with synchronization. From your post I don’t quite see the argumentation with which you try to covince me the command buffer is not actually in use when attempted to free it.

1 Like
3

I expected the bug to be visible in the pseudocode, since there is too much code involved.
I thank you for your advise.
A recent check appeared to prove that the “Transition the layout of the image to a depth attachment” (3) has no effect whatsoever on the error (the error happens even after commenting this part of the code).
You are right, I am not providing enough argumentation, so I am adding it below.

According to the spec, queue submission (vkQueueSubmit) of a command buffer changes its state from Executable to Pending. It must not be modified while in Pending state. Once execution of a command buffer completes, the command buffer either reverts back to Executable or Invalid state. A synchronization command should be used to detect when this occurs (A fence is provided. There is one fence for each possible frame in flight). What I understand is that, when a vkQueueSubmit call is being executed, the command buffer is in Pending state; but when that execution finishes, it goes back to executable state.

The state changes dynamics in my program are the following:

  1. Update UBOs
  2. Create command buffer (CB):
    2.1. vkFreeCommandBuffers(): It moves CB from Recording/Executable to Invalid state.
    2.2. vkAllocateCommandBuffers(): Allocate a new CB (in Initial state) from a command pool.
    2.3. vkBeginCommandBuffer(): It moves CB from Initial to Recording state.
    2.4. Record commands
    2.5. vkEndCommandBuffer(): It moves CB from Recording to Executable state.
  3. vkQueueSubmit(): It moves CB from Executable to Pending state.
  4. Handle window resizing (recreate Swap Chain), if applicable

So, each frame, the command buffer cycle is: Executable > Invalid > Initial > Recording > Executable > Pending > Executable (Repeat cycle). Everything looks correct, that’s why I don’t understand how it’s possible that “the command buffer is in Pending state” when execution reaches vkFreeCommandBuffers() after doing a window resize (which is suppossed not to touch the command buffer). Shouldn’t be there a second thread for this error to be possible?

Is there a way to find out the state of a command buffer? It could help a lot for debugging.

4

Um, no. It destroys command buffers. The buffers do not have a state because they no longer exist. Maybe you meant vkResetCommandBuffers.

Now, vkFreeCommandBuffers can cause other CBs to become invalid. If you have recorded into a primary CB a command to execute a secondary CB, and you free the secondary CB, the primary CB becomes invalid.

But that assumes you didn’t also delete the primary CB.

Also, freeing a CB which has not completed execution makes no sense.

No. You know the state of the command buffer because you did what caused it to be put into that state. Vulkan is not in the habit of telling you what you already know.

5

Your are right, Alfonse_Reinheart, vkFreeCommandBuffer destroys command buffers, it doesn’t puts it in invalid state. I missunderstood the documentation. However, the problem at hand is not there, but I appreciate your respond.

6

Finally, I solved this problem. I could solve it after learning more about Vulkan.
I am using a set of command buffers (one for each swap chain framebuffer, which are 3). Moreover, I am computing 2 frames in parallel. In each frame I was freeing and creating all the CBs. However, I was getting and error (similar to “You cannot free VkCommandBuffer because it’s in use, it’s in pending state”). I was trying to free a VkCommandBuffer that had not finished execution, so it was in pending state. I tried to solve this using vkWaitForFences (wait for one of the CBs to finish execution) but it didn’t work because there was another CB that had not finished execution. Only after replacing vkWaitForFences with vkQueueWaitIdle (wait for all CBs to finish execution) the error was solved.

The following line 1 was replaced by line 2:

  1. vkWaitForFences(e.device, 1, &inFlightFences[currentFrame], VK_TRUE, UINT64_MAX);
  2. vkQueueWaitIdle(e.graphicsQueue);

Facts and learned lessons:

  • Instead of making the whole graphics pipeline to be used only one frame at a time, I was using 2 semaphores for processing 2 frames concurrently. Moreover, I use 3 swap chain framebuffers.
  • vkQueueSubmit: Submits CB. It returns control to the application once queue operations have been submitted. It doesn’t wait for commands to finish execution.
  • There are 2 ways for waiting for CB to finish execution after we call vkQueueSubmit: vkQueueWaitIdle and vkWaitForFences.
  • vkDeviceWaitIdle: Wait for the logical device to finish operations.
  • vkQueueWaitIdle: Wait for operations in a specific command queue to be finished. Wait for all CBs to finish execution.
  • vkWaitForFences: Wait for one (or more) CB to finish execution.
  • vkFreeCommandBuffers: Destroy a CB that is not being used (i.e. in pending state).
7

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.