I know measuring performance is incredibly difficult and problematic. I’m going to use frames per second
I find these statements to be contradictory. If you are willing to use FPS as a performance metric, then I cannot believe that you fully understand how performance measurement works.
Especially since it is easy to use actual time as a performance metric.
They are: Sponza with dynamic lights, Sponza with ambient only, a larger model with ~900,000 triangles and 1 texture, 500+ cubes and 300+ cubes with differing levels of transparency. I’m planning to create a couple more tests at some point to test this further.
OK, so… what do these things actually test?
You can’t just grab a couple of models, render them, measure the performance of that rendering, and then say that those numbers mean something. Performance testing is like a scientific investigation. You have some general notions you want to explore, form falsifiable hypotheses, develop experiments to verify or falsify those hypotheses, perform the experiments, and draw conclusions (with further experiments to come from that).
You just seem to be trying stuff. That’s not going to lead to any meaningful conclusions.
For example, take your “500+ cubes” example. Are these static cubes or are they dynamically moving? How are you making them move dynamically? Are you using instancing or UBO data? How many draw calls are you issuing? Are you doing state changes between those calls, or is it all done through memory? And so forth. Doing each of these will result in different performance results, for different reasons.
Meaningful performance testing is never as simple as “draw a bunch of cubes, see how long it takes”. If you do that, then the numbers you get, even if they’re in time rather than FPS, mean absolutely nothing.
Wouldn’t I only need to multi-thread the creation of them if I was dynamically re-creating them?
Yes. That’s what real applications do. So if you want to have performance numbers that actually mean something to the real world, then you’re going to have to do things as they are done in real applications.
Or at the very least, when doing your performance test, you need to state up-front that you’re using static CBs.
I thought Vulkan would be faster at rendering geometry?
Command buffer-style APIs exist in part because the CPU overhead of immediate APIs had become so great that CPUs were often the bottleneck in large-scale applications.
If all you’re doing is setting up some state and issuing a single draw call to blast something onto the screen, CB APIs should be approximately equivalent in performance to immediate APIs. But that’s not what real-world applications do, so such numbers are essentially useless for comparing the utility of these APIs.