Getting uniform locations from SPIR-V shaders

On my Nvidia card, shaders created with modules loaded from SPIR-V work the same as text. However, on my AMD card the uniforms all seem to be present, but their names are always an empty string. Without the uniform name, how are you supposed to retrieve the uniform location?

GLint countuniforms = 0;
glGetProgramiv(program, GL_ACTIVE_UNIFORMS, &countuniforms);
Uniform u;
int length;
uniforms.reserve(countuniforms);
for (int i = 0; i < countuniforms; i++)
{
	glGetActiveUniform(program, i, buffer.size(), &length, &u.size, &u.type, buffer.data());
	s = std::string(buffer.data());
	if (s.empty()) continue; //this is always an empty string?
	u.index = glGetUniformLocation(program, s.c_str()); //so how to call this?
	if (u.index < 0) continue;
	uniformindexes[s] = u.index;
	if (u.index >= uniforms.size()) uniforms.resize(u.index + 1);
	uniforms[u.index] = u;
}

See issue 23 in ARB_gl_spirv:

By “offline reflection” are they saying the uniform information needs to be manually added to a JSON file and loaded separately, or are they talking about parsing the SPIR-V data?

How do you confirm that the uniforms are of the correct size and type? Does glGetActiveUniform accept the uniform location as the index when SPIR-V is used? This is not how it works with normal text shaders.

“Needs to be” is a strong phrase. What it’s saying is that, if your program represents uniforms by name instead of by locations, you will need to handle the mapping from name to location yourself. That is, the OpenGL uniform interface accesses uniforms by locations. If you want to access them by names, you can’t use glGetUniformLocation; you have to build and use a mapping of name to location yourself.

How you do that is up to you. Or you can just switch to using locations directly, putting the mapping outside of your program.

Why would you need to confirm that? That kind of confirmation generally ought to happen at the tool level or outside the runtime of the application.

In any case, the introspection API largely still works with SPIR-V-based shaders. Uniforms in a program are still put into an array accessed by uniform indices. And the introspection API can be used to query properties about any uniform in that index range.

What you can’t do is translate the name into anything. You can talk about uniform index X. You can search all of the uniforms to find a uniform index whose location is Y. But you cannot ask for the location or index by a name. You may search all uniforms for a specific name, but there’s no guarantee that this will work either (since the compiler can choose not to bring names along, even if they’re in the SPIR-V).

Locations and indices are still different things, and you can’t query an index by location. You have to scan all of the uniforms to find the one with the location of interest (or build a mapping table).

The reason I prefer to have easy access to named uniforms is that custom shaders are a feature that is exposed to end users. Asking them to manually define all their uniforms in a separate file or to remember constants for the values they want to set degrades the user experience.

Using SPIRVCross, I can get the name and location of each uniform. How do I retrieve the type and size?

int sz = data->GetSize();
sz = Ceil(float(sz) / 4.0f);
std::vector<uint32_t> spirv_data(sz);
memcpy(spirv_data.data(), data->Data(), data->GetSize());
spirv_cross::CompilerGLSL compiler(spirv_data);

// Get the reflected resources
spirv_cross::ShaderResources resources = compiler.get_shader_resources();

// Retrieve uniform information
RenderShader::Uniform u;
for (const auto& uniform : resources.gl_plain_uniforms)
{
	auto type = compiler.get_type(uniform.type_id);
	u.name = compiler.get_name(uniform.id);
	u.index = compiler.get_decoration(uniform.id, spv::DecorationLocation); 			
	u.size = ???
	switch (type.basetype)????
	{
		case ???:
			u.type = ???
	}
}

Here is most of the answer. The SPIRType structure seems to be hidden, so I don’t know where the constants for this are declared:

switch (type.basetype)
{
case 7: //integer
				switch (type.vecsize)
				{
				case 1:
					u.type = GL_INT;
					break;
				case 2:
					u.type = GL_INT_VEC2;
					break;
				case 3:
					u.type = GL_INT_VEC3;
					break;
				case 4:
					u.type = GL_INT_VEC4;
					break;
				}
				break;
case 8: //unsigned integer
				switch (type.vecsize)
				{
				case 1:
					u.type = GL_UNSIGNED_INT;
					break;
				case 2:
					u.type = GL_UNSIGNED_INT_VEC2;
					break;
				case 3:
					u.type = GL_UNSIGNED_INT_VEC3;
					break;
				case 4:
					u.type = GL_UNSIGNED_INT_VEC4;
					break;
				}
				break;
case 13: //float
				switch (type.vecsize)
				{
				case 1:
					u.type = GL_FLOAT;
					break;
				case 2:
					u.type = GL_FLOAT_VEC2;
					break;
				case 3:
					u.type = GL_FLOAT_VEC3;
					break;
				case 4:
					u.type = GL_FLOAT_VEC4;
					break;
				}
				break;
}

I am able to retrieve the uniform info now, but glProgramUniform1f is generating an invalid operation error. The type, location, and size being used match what they should be in the shader. However, the value appears to be set correctly, and rendering is fine once the values are initially set.

Are there any strange rules with SPIR-V that would cause this?

	void RenderShader::SetUniform(const int index, const float f)
	{
		if (index < 0 or index >= uniforms.size()) return;
		auto& u = uniforms[index];
		if (u.type != GL_FLOAT) return;
		if (u.size != 1) return;
		if (u.defined and f == u.value.i.x) return;
		glCheckError();
		glProgramUniform1f(program, u.index, f);
		glCheckError();
		u.defined = true;
		u.value.i.x = f;
	}

Uniform indices and uniform locations aren’t the same thing.

For the purposes of introspection, all uniforms in a program object are arranged in a single array. The index of a uniform in that array is the uniform’s index. Indices are contiguous; give the maximum number of uniforms in a shader, every index from [0, max) will represent a distinct uniform.

Among the properties of a uniform (type, number of components, etc) is its location. This is the value you give to glProgramUniform* functions to identify which uniform you’re talking about. Uniforms that are part of UBOs have no location. Uniform locations are specified by shaders and are not (necessarily) contiguous.

So a uniform’s index is arbitrarily determined by the OpenGL shader compiler/linker, while the location is provided by the user. So either your index parameter is misnamed, or you should be passing the uniform’s location.

Similarly:

u.index = compiler.get_decoration(uniform.id, spv::DecorationLocation);

The index member is misnamed. You cannot use that index to ask for any information from the compiled and linked program.

Well yeah, “index” is an unfortunate name for the member, but I think I am still retrieving the information correctly:

Uniform u;
for (const auto& uniform : resources.gl_plain_uniforms)
{
	auto type = compiler.get_type(uniform.type_id);
	u.location = compiler.get_decoration(uniform.id, spv::DecorationLocation);				
	if (u.location < 0) continue;
	u.name = compiler.get_name(uniform.id);
	if (u.name.empty()) continue;
	u.size = 1;
	auto thing = type.basetype;
	switch (type.basetype)
	{
	case 7: //integer
		switch (type.vecsize)
		{
		case 1:
			u.type = GL_INT;
			break;
		case 2:
			u.type = GL_INT_VEC2;
			break;
		case 3:
			u.type = GL_INT_VEC3;
			break;
		case 4:
			u.type = GL_INT_VEC4;
			break;
		}
		break;
	case 8: //unsigned integer
		switch (type.vecsize)
		{
		case 1:
			u.type = GL_UNSIGNED_INT;
			break;
		case 2:
			u.type = GL_UNSIGNED_INT_VEC2;
			break;
		case 3:
			u.type = GL_UNSIGNED_INT_VEC3;
			break;
		case 4:
			u.type = GL_UNSIGNED_INT_VEC4;
			break;
		}
		break;
	case 13: //float
		switch (type.vecsize)
		{
		case 1:
			u.type = GL_FLOAT;
			break;
		case 2:
			u.type = GL_FLOAT_VEC2;
			break;
		case 3:
			u.type = GL_FLOAT_VEC3;
			break;
		case 4:
			u.type = GL_FLOAT_VEC4;
			break;
		}
		break;
	}
	uniforms.push_back(u);
}

Also, you have to do this to make sure you are only retrieving the active uniforms:

			auto vars = compiler.get_active_interface_variables();
			for (const auto& uniform : resources.gl_plain_uniforms)
			{
				if (vars.find(uniform.id) == vars.end()) continue;// only process active uniforms