Is it possible to “link” an existing compiled .spv file into a shader program when it is being compiled? Like, can I compile “myshader.frag” and link in some functions from “utilities.spv” without the source code for the utilities.spv file? (I’m using GLSLValidater for compiling.)
I’m not sure what you mean by this.
Every SPIR-V compilation has to be closed, much like C and C++. External code can be linked post-compilation, but any declarations used by a piece of SPIR-V code must be visible in that code.
So if you intend for some shader to reference code outside of that shader, it must have a declaration inside of that shader which matches one defined by the other shader. You can then link the two together later.
Okay, I have a file called “test.frag.glsl” with a function declared inside it like this:
vec4 testfunc()
{
return vec4(1,0,0,1);
}
I am trying to compile the file with this command line:
glslangValidator.exe "test.frag.glsl" -o -V "test.frag.spv"
However, the -V option turns on the -l option, which links the file and expects to see a main() function:
ERROR: Linking fragment stage: Missing entry point: Each stage requires one entry point
I’m expecting to compile this file without a main() function, and then my next step is to link this together with a module that does have a main function, like this:
glslangValidator.exe "PBR/PBR_intermediate.frag.spv" "test.frag.spv" -l "PBR_final.frag.spv"
It looks like this is not currently supported for Vulkan.
Not sure of the specifics of your situation, but would use of specialization constants be useful to you for post-compile tweaks like this?
That is, use your utilities code in the main shader(s) by default in 1 or more code paths, selected by a specialization constant. And then post-compile, tune the specialization constant to determine what code path(s) used within it (if any) are preserved in the final shader.
With this, the initial compile to SPIR-V is more heavy-weight. But the variation customization is then all done in the IR or lower level (like I think you’re trying to achieve with linking in precompiled SPIR-V modules).
I just wanted to see if I could hide some bits of code while still allowing the end user to implement custom shaders.
What you want is possible, but my previous post was an over-simplification of the process and contained some misinformation.
From a purely language standpoint, SPIR-V can do what you want. Normally, a SPIR-V module must have at least one OpEntryPoint declared in it. However, if the module is built with the Linkage capability active, then it doesn’t need one.
The Linkage capability allows you to specify that certain functions or variables are exported from the module or imported to the module. This allows you to have a main module that calls into some other module.
The problem is that neither Vulkan nor OpenGL allows you to do this at the API level. In OpenGL glSpecializeShader requires you to specify an entry point name, which means that the module must have one. Vulkan’s vkCreateShaderModule itself does not require specifying an entrypoint name, but every API that consumes a compiled module does. For example, VkPipelineShaderStageCreateInfo requires an entry point name. And since this structure only references a single SPIR-V module, there’s no way to use a module that doesn’t have an entrypoint.
That being said, there is an alternative way to handle this: manually. Basically, what you have to do is manually combine two (or more) SPIR-V modules into a single SPIR-V module.
Now, SPIR-V as a format and language is actually pretty well-designed for mechanical concatenation. Each SPIR-V module references things by index, and every SPIR-V module has a value specifying the largest index that the module uses. So if you want to add more stuff to the end of the module, you can just increment all of the added indices by that largest index value (and compute a new largest index for your combined module, of course). And while there are some ordering constraints that will require you to stuff some things into the module of the combined module, overall this is a simple mechanical process.
Now, if you do that naively as I outlined, this means that nothing from one module can talk to the other. Everything is connected by indices, so the combined module would have two disjointed sets of indices. So you’ll need another system for allowing code from one module to talk to code from another. This can be done by matching function signatures and names (basically assuming that if two names and their attendant definitions are exact matches, then they’re the same thing), and this too can be a mechanical process.
The SPIRV-Tools project claims to support linking together binaries. I have no idea how well that works, and, as with most things, documentation is scarce.
The bigger issue is the GLSLang compiler itself. Finding documentation on its available options isn’t easy, but since GLSL says that it is OK for a file to not have main in it, it should be possible to compile such a thing. However, it’s not clear if glslang can handle that.