SPIRV - automatic uniform detection?

Hello guys,

I watched an game engine tutorial where somebody used openGL and implemented an automatic uniform detection & binding system.
Vulkan uses (binary) precompiled SPIR-V.
My Question is, if its possible to detect uniforms on runtime in the spir-v text and for example automatically create the appropriate pipeline layout based on which uniforms the shader is using?

Thanks in advance
SH

Yes, you could (FYI: SPIR-V is defined in binary, not text).

A better question would be… why would you want to?

What would you do with that? If you have two shaders, and you have absolutely no idea what goes into them, then how exactly do you plan to build descriptor pools with them? How do you plan to attach textures to them when you have no idea what a particular descriptor means? Is it a diffuse texture? Is it a normal map? Is it something else entirely?

You don’t know. You can’t know purely from the descriptor information. And if you don’t know what a particular descriptor means, how can you assign it appropriate data?

This doesn’t even make sense in GLSL, since it doesn’t give you any additional information. The only thing it gives you that SPIR-V won’t is a name (names are not required for things in SPIR-V). Unless you intend to use that name to decide on the meaning of the texture, this technique is equally dysfunctional in GLSL. And if you do intend to use the name… why bother with this process at all? Just query the uniform location by name with OpenGL’s APIs.

This whole process makes no sense, unless you’re writing a system where the same person who provided you the shader will later tell you what to assign to those descriptors. And that’s generally not how people do things.

Good use of Vulkan involves discipline. You know what your shaders say, and you know how to use them. You know what those descriptors mean, and you know what to do with them. Set 0 has a specific meaning to you, as does set 1 and set 2. Descriptor 5 in set 2 means something to your code.

Such systems do not play well with shaders that do what they want and expect you to work out what they really mean.

Yes you are right. Currently i have for every rendering-pass (VertexShader + FragmentShader) a subclass from a “ShaderBase” class where i define all the descriptor sets and update them. This is kind of annoying and i try to find another way to do this.
It would be optimal to find a way to change/add minimal in code when writing a new shader

Currently i have for every rendering-pass (VertexShader + FragmentShader) a subclass from a “ShaderBase” class where i define all the descriptor sets and update them.

… why?

When I said that you needed a disciplined approach, I meant that. You need to define what layout you want to use, then make shaders that conform to that layout. If a particular shader doesn’t need certain descriptors, then it doesn’t have to use them, but the pipeline you build would still include them in the layout.

Now, there are times when you need multiple layouts. But layouts should not be associated with specific shaders, but with specific kinds of rendering.

For example, if you’re doing deferred rendering, you have your geometry pass, your lighting passes, blended stuff, and maybe tone mapping, bluring, SSAO and other full-screen operations. Each one of those groups of things would probably have its own pipeline layout, but all shaders used within them would share the same layout. Maybe your geometry pass has two layouts: one for objects that use bone skinning and one for others. But little more than that.

You should not be frequently changing between pipelines that use different layouts. And it certainly should not be per-object or even per-shader.

As for the contents of descriptor sets themselves, those too need to be separate from shaders. Ideally (given the appropriate hardware features), you should be rendering lots of objects that use the same descriptor sets.

I had kind of this approach before but i always got an error message from the debug layer about “disturbed descriptor sets”.
Back to my original question, i found something interesting in this presentation: https://www.khronos.org/assets/uploads/developers/library/2016-vulkan-devday-uk/4-Using-spir-v-with-spirv-cross.pdf
On Page 64 about Pipeline-Layouts:
"Need to figure this out automatically, or write every layout by hand

  • Latter is fine for tiny applications
  • Vulkan does not provide reflection here, after all, this is vendor neutral information"

So there is already a nice library “SPIRV-Cross” which detects the descriptor sets in spirv and could be used to automatically create an pipeline-layout.
That’s exactly what i was looking for and i will try to use this

I had kind of this approach before but i always got an error message from the debug layer about “disturbed descriptor sets”.

Then you were probably changing descriptor sets in the wrong order.

That’s exactly what i was looking for and i will try to use this

I’m still curious as to how you would use that information.

So you build a pipeline layout from a collection of SPIR-V modules. OK, so… how do you know what data goes where?

You know that descriptor 4 in set 1 has a UBO. Do you know what that UBO stores? You know that descriptor 8 in set 0 is an array of 2D textures. Do you know how those textures are used in your shader?

Your application needs to know what those descriptors mean. Which means that your application has to know that set 1, descriptor 4 contains the per-object matrices used to render the object. And if your application knows that, then your application is perfectly capable of building pipeline layouts without asking the SPIR-V modules about them.

sounds like you want spir-V introspection and additional features with descriptor sets and layouts.

You could create a glsl compilation/spir-V introspection library that mimics the opengl shader compilation and deal with the layouts (either generating them and exposing what it generated or making it conform to a provided layout). But it will never be part of the driver implementation.

It should also be noted that introspecting a single SPIR-V shader and building a pipeline layout from that is a bad idea. The reason being that in many occasions, shaders for one set of meshes may not use all the descriptors that shaders for another set of meshes does. This would result in two separate pipeline layouts. So now your pipeline change also means a pipeline layout change.

That’s bad.

Thanks for all of your answers, i will keep this in mind and hopefully develop a system what makes me happy :slight_smile: