Timestamp query result never available?

I’m trying to use a simple timestamp query, but I’ve once again ran into a bit of trouble.
Here’s my test-code which I used in the cube-demo from the SDK at the top of the ‘demo_draw’-function (Executed every frame):


static void demo_draw(struct demo *demo) {
	static VkQueryPool queryPool = NULL;
	const uint32_t queryId = 0;
	if(queryPool == NULL)
	{
		VkQueryPoolCreateInfo info;
		info.flags = 0;
		info.pipelineStatistics = 0;
		info.pNext = NULL;
		info.queryCount = 1;
		info.queryType = VK_QUERY_TYPE_TIMESTAMP;
		info.sType = VK_STRUCTURE_TYPE_QUERY_POOL_CREATE_INFO;

		VkResult r = vkCreateQueryPool(demo->device,&info,NULL,&queryPool);
		assert(!r);

		vkCmdResetQueryPool(demo->buffers[demo->current_buffer].cmd,queryPool,queryId,1); // "As with other queries, the query must be reset using vkCmdResetQueryPool before requesting the timestamp value be written to it."
		vkCmdWriteTimestamp(demo->buffers[demo->current_buffer].cmd,VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT,queryPool,queryId); // Tried it with different pipeline stages; Same result regardless
	}

	// From the specs: "The first query’s result is written starting at the first byte requested by the command, and each subsequent query’s result begins stride bytes later."
	// "If VK_QUERY_RESULT_WITH_AVAILABILITY_BIT is used, the final element of each query’s result is an integer indicating whether the query’s result is available, with any non-zero value indicating that it is available."
	uint32_t resultData[2] = {0,0}; // resultData[0] = Query result; resultData[1] = Availability
	VkResult r = vkGetQueryPoolResults(demo->device,queryPool,queryId,1,sizeof(uint32_t) *2,resultData,sizeof(uint32_t) *2,VK_QUERY_RESULT_WITH_AVAILABILITY_BIT);
	assert(!r);
	if(resultData[1] != 0)
		printf("Result available
");
	else
		printf("Result not available
");
	[...] // Rest of the rendering code as usual
}

resultData[1] is always 0, so the result is never available.

Something that strikes me as odd is this part from the specs:

So, if I understand this correctly, since I only have one query in this case, if the query is unavailable, resultData[1] should be set to 0 and the return value should be VK_NOT_READY, correct?
Well, resultData[1] is being set to 0 (I’ve checked), however the return value is always VK_SUCCESS. So… The query is in the available state, but at the same time it’s not?:confused:

Are there any examples around anywhere on how to use timestamp queries properly?

I have a small demo that writes (and displays) timestamps for the different parts of the pipeline which I haven’t put online yet, but it’s working fine and I’m getting time stamps as expected.

The demo initially uses VK_QUERY_RESULT_WAIT_BIT to make sure the query results are available before displaying, but it also works fine with VK_QUERY_RESULT_WITH_AVAILABILITY_BIT.

From your code it’s not quite clear when you get the results. Do you fetch them right after the write?

vkGetQueryPoolResults is called every frame, starting right after the write, however the result never becomes available, no matter how long I keep the program running (It should be available the next frame anyway).
I’ve uploaded the complete demo here. It’s 1:1 the same as the regular cube demo, except for the part I’ve marked with “// Timestamp Query Start” to “// Timestamp Query End”.

I’ve also tried pipeline statistics queries, from which I was able to retrieve the results the same way just fine. Only timestamp queries are problematic for some reason.

[QUOTE=Sascha Willems;40232]I have a small demo that writes (and displays) timestamps for the different parts of the pipeline which I haven’t put online yet, but it’s working fine and I’m getting time stamps as expected.

The demo initially uses VK_QUERY_RESULT_WAIT_BIT to make sure the query results are available before displaying, but it also works fine with VK_QUERY_RESULT_WITH_AVAILABILITY_BIT.[/QUOTE]
Your demos have helped me quite a bit in the past, so I’ll be looking forward to it. :slight_smile:
It’s there an eta?

[QUOTE=Silverlan;40233]vkGetQueryPoolResults is called every frame, starting right after the write, however the result never becomes available, no matter how long I keep the program running (It should be available the next frame anyway).
I’ve uploaded the complete demo here. It’s 1:1 the same as the regular cube demo, except for the part I’ve marked with “// Timestamp Query Start” to “// Timestamp Query End”.

I’ve also tried pipeline statistics queries, from which I was able to retrieve the results the same way just fine. Only timestamp queries are problematic for some reason.[/QUOTE]

My example gets the result after ending the render pass and submitting the command buffer to the queue. Maybe that’s the difference. Did you try without the vkCmdResetQueryPool?

Thx :slight_smile:

Nope, so little time so much to do. I only get to work on Vulkan in my spare time. But maybe this or next weekend. If you’re interested I can upload the work-in-progress code sometime this week.

Well, the specs explicitly state that “After query pool creation, each query is in an undefined state and must be reset prior to use. Queries must also be reset between uses. Using a query that has not been reset will result in undefined behavior.”.
Either way, I tried it without resetting just now, but the result is the same (No query result available).

If it’s no bother, that would be really helpful. :slight_smile:

I modified your program to get the timestamp queries working. You had the query part right.

Move the vkCmdResetQueryPool and vkCmdWriteTimestamp calls out of the if (queryPool == NULL) block and put them between the demo_set_image_layout() and demo_flush_init_cmd() calls. Also change the commandBuffer arg in both to just demo->cmd. I don’t think that you were recording commands in the old location.

You can leave the block of code that queries the pool results in the same place, but it will assert on the first pass since there was no submit at that point on the first pass. It might make more sense to move this block to the bottom of demo_draw() after the WaitIdle, when a submit has been completed and you can actually expect to find a query result. Note that vkGetQueryPoolResults can return VK_NOT_READY as well as VK_SUCCESS, so that assert should be changed to pass for either (or removed).

I also changed the print statement to print the timestamp value, and could see it change with each frame.

[QUOTE=karlschultz;40236]
Move the vkCmdResetQueryPool and vkCmdWriteTimestamp calls out of the if (queryPool == NULL) block and put them between the demo_set_image_layout() and demo_flush_init_cmd() calls. Also change the commandBuffer arg in both to just demo->cmd. I don’t think that you were recording commands in the old location.

You can leave the block of code that queries the pool results in the same place, but it will assert on the first pass since there was no submit at that point on the first pass. It might make more sense to move this block to the bottom of demo_draw() after the WaitIdle, when a submit has been completed and you can actually expect to find a query result. Note that vkGetQueryPoolResults can return VK_NOT_READY as well as VK_SUCCESS, so that assert should be changed to pass for either (or removed).[/QUOTE]
Thanks, that did the trick.
However, I did notice something strange when using more than one query:


static void demo_draw(struct demo *demo) {
	static VkQueryPool queryPool = NULL;
	if(queryPool == NULL)
	{
		VkQueryPoolCreateInfo info;
		info.flags = 0;
		info.pipelineStatistics = 0;
		info.pNext = NULL;
		info.queryCount = 2;
		info.queryType = VK_QUERY_TYPE_TIMESTAMP;
		info.sType = VK_STRUCTURE_TYPE_QUERY_POOL_CREATE_INFO;

		VkResult r = vkCreateQueryPool(demo->device,&info,NULL,&queryPool);
		assert(!r);
	}

	[...]

    // Assume the command buffer has been run on current_buffer before so
    // we need to set the image layout back to COLOR_ATTACHMENT_OPTIMAL
    demo_set_image_layout(demo, demo->buffers[demo->current_buffer].image,
                          VK_IMAGE_ASPECT_COLOR_BIT,
                          VK_IMAGE_LAYOUT_PRESENT_SRC_KHR,
                          VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL,
                          0);

	vkCmdResetQueryPool(demo->cmd,queryPool,0,1); // Reset query #1
	vkCmdWriteTimestamp(demo->cmd,VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT,queryPool,0); // Write query #1
	vkCmdResetQueryPool(demo->cmd,queryPool,1,1); // Reset query #2
	vkCmdWriteTimestamp(demo->cmd,VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT,queryPool,1); // Write query #2

    demo_flush_init_cmd(demo);

    [...]

    err = vkQueueWaitIdle(demo->queue);
    assert(err == VK_SUCCESS);

	uint32_t resultData[2] = {0,0};
	VkResult r = vkGetQueryPoolResults(demo->device,queryPool,0,1,sizeof(uint32_t) *2,resultData,sizeof(uint32_t) *2,VK_QUERY_RESULT_WITH_AVAILABILITY_BIT); // Retrieve result and availability of query #1
	if(resultData[1] != 0)
		printf("Result available %i
",resultData[0]);
	else
		printf("Result not available
");

    vkDestroySemaphore(demo->device, presentCompleteSemaphore, NULL);
}

If I run this, the result will, once again, never be available. However, if I comment the line which resets query #2 (vkCmdResetQueryPool(demo->cmd,queryPool,1,1);), the result of query #1 becomes available just fine.
Why is that? Resetting query #2 should have no effect on query #1 whatsoever, right? (https://www.khronos.org/registry/vulkan/specs/1.0/apispec.html#vkCmdResetQueryPool)

Here’s the entire cube.c file for the demo.

I didn’t really try to duplicate your problem exactly. I moved to the latest version of cube.c, which changed a lot of the code in demo_draw(). I’m thinking that wedging the timestamp writes in there isn’t really the right way to do it and may be causing your issue. So I applied your changes the “right” way by creating the query pool in demo_create_device() and adding the pool reset and write timestamp cmds in demo_draw_build_cmd().

But I added the second query pool reset and the second timestamp write, just like you did, and have no problems with the available flag. I also added QueryPoolResults calls to query each of the timestamps independently and then again in a single query.

Modified program is here.

Are both results available for you? I just tried your demo, and query #2 does become available, but query #1 doesn’t. It looks like the same behavior as in my demo, query #1 only becomes available if I comment out the reset for query #2.

Here’s a video of what I mean.

Yes, both results were available, with no lines commented out.

This could be a problem/difference in drivers. My working result is on Ubuntu 15.10 with NVIDIA 364.16 drivers.

I have an AMD card, so a driver bug seems possible.
I had two more people test the demo, one with an AMD GPU, and another on NVIDIA (Both on Windows). Same results, it worked on NVIDIA, but didn’t on AMD. I’ll ask about it on the AMD forums.

Can’t edit my previous post, just wanted to say that it was indeed a driver bug, and it’s probably going to be fixed in one of the next AMD driver updates: