UBO get size of arrays_of_arrays

[N247S]

Hey everyone,

I was wondering if its possible to get the length of an array containing arrays?
Im currently trying to accomplish this with UBO’s, though I think this problem also relates to any interface block, or normal uniform variables.

In GLSL its possible to have something like this:

uniform float[2][5][3] arrays;

When you try to query the size with ‘glGetActiveUniformi’ you’ll get the total size of the array (in this case 253=30),
but there is no way (as far as I can find) to split this multidimensional array in parts, or get the size of a nested array.

I tried to work around this by querying the uniformId for the full length of the array where I used “arrays[3]”
for example to check if that specific index exist. If not, than we have a nested array.
The problem is, for UBO’s you cannot query an uniformId that way. Since UBO’s have a separate Id system.
Unfortunatly from querying the UBO indices I get only one id/index for the whole multidimensional array meaning this is a dead end.

any help or direction is appreciated,
N247S

[Alfonse_Reinheart]

When you try to query the size with ‘glGetActiveUniformi’ you’ll get the total size of the array (in this case 253=30)

It shouldn’t.

The OpenGL specification is very clear on this. An array of arrays works identically to an array of structs. Therefore, your arrays definition ought to result in 10 separate uniforms, named:

arrays[0][0][0]
arrays[0][1][0]
arrays[0][2][0]
arrays[0][3][0]
arrays[0][4][0]
arrays[1][0][0]
...
arrays[1][4][0]

Each of these variables will be an array of 3 floats. If you’re getting 30 out of OpenGL, then your implementation is broken.

The problem is, for UBO’s you cannot query an uniformId that way

Sure you can. You just have to name it correctly. The reason why arrays[3] didn’t work is because there’s no such variable. And not just because the arrays first index was 2

The variables are named as above, possibly prefixed in accord with interface block member names.

[N247S]

Thanks for the reply!

I hope its clear that the problem is comming up with UBO’s since there are a couple things that are not they way you described.
First the multidimensional array is treated as one variable. (and yes it does return 30 with ‘GL31.glGetActiveUniformi’)
Second the example ‘array[3]’ would indeed return -1, but any (possible) valid name returns the exact same result.
Here is a little bit better example which might clarify things.

#version 430

layout (std140) uniform myUBO
{
    float[2][5][3] array;
};

void main()
{
    // Calcs stuff goes here.
}

Here are some snippits of the querying.

// returns -1;
int id = GL20.glGetUniformLocation(programID, "myUBO.array[0][0][0]");


// returns  30. (ps. 'array_UBO_ID' is the id queried from myUBO using
// GL31.glGetActiveUniformBlock with parameter GL31.GL_UNIFORM_BLOCK_ACTIVE_UNIFORM_INDICES)

int size = GL31.glGetActiveUniformsi(programID, array_UBO_ID, GL31.GL_UNIFORM_SIZE));

there is a litle more to the naming stuff which might throw you off.
“GL31.glGetActiveUniformName” returns “myUBO.array[0]”.
But using that String with "Gl20.glGetUniformLocation will return -1;

I tried every possible combination (with and without interface prefix) to get any uniformLocation before I posted this,
so Im pretty sure you are not able to get a uniformLocation for an separate (UBO) array.

Thanks again for your answer, I hope we are still on the same page regarding the problem
N247S

[Alfonse_Reinheart]

Here is a little bit better example which might clarify things.

array[2][5][3] lacks a type declaration and is therefore not legal.

It’s also important to recognize that there’s a difference between float array[2][5][3] and float[2][5][3] array.

Can you post the actual GLSL shader and the actual code you use to introspect it?

I tried every possible combination (with and without interface prefix) to get any uniformLocation before I posted this, so Im pretty sure you are not able to get a uniformLocation for an separate (UBO) array.

And yet, the standard says that it’s supposed to work as I described. So either your code is wrong or your OpenGL implementation is wrong.

Since you’re using GL 4.3 or greater, try using this code that uses the “new” introspection API to query every uniform block’s members.

[N247S]

Such a minor detail…, sorry for that.
Anyway I updated it, it should have been “float[2][5][3] array;”.

I actually tried to provide the minimal version possible of what I am using for testing.
(there are a lot of comments, but the non-comented stuff is the same)
e.g. a fragment shader snipit, and a snippit of what I use to query stuff.

so a full version could be:

#version 430
//Vertex Shader

    uniform mat4 projection_matrix;
    uniform mat4 modelview_matrix;
    
    attribute vec4 vertex;
    
void main()
{
    gl_Position = projection_matrix * modelview_matrix * vertex;
}

#version 430
//Fragment Shader

layout(std140) uniform myUBO
{
    float[2][5][3] arrays;
};

void main()
{
    gl_FragColor = vec4(myUBO.arrays[0][0][0], myUBO.arrays[0][0][1], myUBO.arrays[0][0][2], 1.0);
}

with the java snipit

// coverts the files to strings, creates, and links the shaders/program etc. etc.
int programID = CreateProgram();

//...
Init stuff for the Vertex Shader
...//

// This is enough to get the results I get.
int blockIndex = GL31.glGetUniformBlockIndex(programID, "myUBO");
IntBuffer indices = ByteBuffer.allocateDirect(16 << 2).order(ByteOrder.nativeOrder()).asIntBuffer(16);
GL31.glGetActiveUniformBlock(programID, blockIndex, GL31.GL_UNIFORM_BLOCK_ACTIVE_UNIFORM_INDICES, indices);


// the indices buffer only contains one value at this point, but for confirmation I print everything
while(indices.hasRemaining())
    System.out.println(indices.get());

// resset the position because of the printing.
indices.position(0);

// since there is only one index, I use the non-buffer variant of the query methods.
int index = indices.get();

System.out.println(GL31.glGetActiveUniformsi(pr.getProgram(), index , GL31.GL_UNIFORM_OFFSET));// prints 0
System.out.println(GL31.glGetActiveUniformsi(pr.getProgram(), index , GL31.GL_UNIFORM_TYPE));// prints 5126 (glEnum GL_FLOAT)
System.out.println(GL31.glGetActiveUniformsi(pr.getProgram(), index , GL31.GL_UNIFORM_SIZE));// prints 30
System.out.println(GL31.glGetActiveUniformsi(pr.getProgram(), index , GL31.GL_UNIFORM_ARRAY_STRIDE));// prints 16
int nameLength = GL31.glGetActiveUniformsi(pr.getProgram(), index , GL31.GL_UNIFORM_NAME_LENGTH);
System.out.println(nameLength );// prints 16
System.out.println(GL31.glGetActiveUniformName(pr.getProgram(), index, nameLength);//  prints myUBO.arrays[0]

// This is the GL4.3 variant of the above, resulting in the same results. I tried to play around with the index as well, which all returns -1; (since there is no other variable)
System.out.println(GL43.glGetProgramResourceName(pr.getProgram(), GL43.GL_UNIFORM, index, 30));//  prints myUBO.arrays[0]

Thanks for the tip on GL4.3, although Im afraid it results in the same behavior.
I hope this is enough to go on?

Thanks again,
N247S

[Alfonse_Reinheart]

I hope this is enough to go on?

… in a sense, yes, but not the way you want.

This:

myUBO.arrays[0][0][0], myUBO.arrays[0][0][1], myUBO.arrays[0][0][2]

Is a compile error. The name of variables in an interface block, on the GLSL side, are never prefixed with the block name. If you give the block an instance name, then they are prefixed with that instance name. But myUBO is the block name, not the instance name. So to GLSL, the name of that variable is simply arrays, not myUBO.arrays.

If you feed that shader into a compiler, it ought to fail to compile, with an error pointing to main. By all rights, none of those introspection functions should be expected to return reasonable values. So until you get the compile error sorted out, you can’t expect to properly introspect your shaders.

Always check your compiler/linker errors.

Also, I find this suspect:

Init stuff for the Vertex Shader

What about the fragment shader? That’s where the UBO is, after all. Do you compile and link it in? Do you check for errors?

[N247S]

I got the feeling things are mixed up here.

First the prefix is defenitly needed, and yes I know its the blockName. thats what makes it a UniformBufferObject instead of a structure or a BufferObject.
I do check for errors while compiling etc. and for confirmation, it does throw a compile error when I DON’T use the prefix.
After all, “glGetActiveUniformName” returns a name including the prefix as well. Furthermore if I use a instance name like:

layout (std140) uniform Block
{
    // variables
} myUBO;

It doesn’t recognize the interface block as an UBO, and “glGetUniformBlockIndex” would return -1;

And about the last comment, yes I do init the fragment shader After the main snippit I posted. for readability I cut out the vertex stuff, since it has nothing to do with the current problem.
Besides this problem is not about the initialization stuff, but about the querying stuff.
the “createProgram” method represents the Shader compiling and linking/attaching plus the program creation and linking, which doesn’t throw any errors and is thoroughly tested.

So here is the problem:
How do you query the correct (sub) length of nested arrays?

Again thanks for the reply,
N247S

[Alfonse_Reinheart]

First the prefix is defenitly needed, and yes I know its the blockName.

No, it is not needed:

If an instance name (instance-name) is not used, the names declared inside the block are scoped at the global level and accessed as if they were declared outside the block. If an instance name (instance-name) is used, then it puts all the members inside a scope within its own name space, accessed with the field selector ( . ) operator (analogously to structures).

The specification cannot be more clear on this. This statement in the spec is followed by examples:

in Light {
vec4 LightPos;
vec3 LightColor;
};
in ColoredTexture {
vec4 Color;
vec2 TexCoord;
} Material; // instance name
vec3 Color; // different Color than Material.Color
vec4 LightPos; // illegal, already defined
...
... = LightPos; // accessing LightPos
... = Material.Color; // accessing Color in ColoredTexture block

These examples may use in interface blocks, but uniform interface blocks work the exact same way in this regard.

The specification goes on to say:

Outside the shading language (i.e., in the API), members are similarly identified except the block name is always used in place of the instance name (API accesses are to shader interfaces, not to shaders). If there is no instance name, then the API does not use the block name to access a member, just the member name.

Again, this cannot be more clear. So if this is not the behavior you are seeing, then either your code is buggy or the implementation is buggy. But there is no question as to what the OpenGL specification says should happen.

It doesn’t recognize the interface block as an UBO, and “glGetUniformBlockIndex” would return -1;

And which name were you trying to query when you used that UBO definition? Because if you were trying to query “myUBO”, that shouldn’t work. The name of the block in that declaration is “Block”, not “myUBO”.

And about the last comment, yes I do init the fragment shader After the main snippit I posted. for readability I cut out the vertex stuff, since it has nothing to do with the current problem.
Besides this problem is not about the initialization stuff, but about the querying stuff.

Bugs love hiding in places where you think they have “nothing to do with the current problem.”

If you do not initialize your program correctly, you cannot expect to get correct behavior from querying stuff from it. And the behavior you appear to be getting is so incorrect that I can’t imagine an implementation is this buggy.

How do you query the correct (sub) length of nested arrays?

You can’t. Even ignoring driver bugs and the like, the specification does not allow you to simply query the sizes of arrays of arrays.

As I stated in my very first post, arrays of arrays are treated like arrays of structs. The example you originally posted should have 10 uniforms in that block, with each uniform being a 3-element array of floats. You can query each of the 10 uniforms of course, and you can do some processing to try to deduce the sizes of them based on that. But you cannot simply ask for the size of an array of arrays.

[N247S]

Thanks for your reaction.

Your last comment made me worry about something. I have a lot of problems with programs with DirectX as renderengine on my current pc.
But I have never tested it fully with opengl, and from the first few tests it seems that shaders are verry buggy on this system which might explain the wierd behavior.

I have a couple misstakes in the testing, which explains a couple of unexpected results, though the following are still standing;

On my current system I can refer to a UBO variable with and without the use of a Block-name (for UBO’s without an instance-name)

As you mentioned several times, arrays-of-arrays are treated like structures which is true. The problem is that every variable has a program-location and program-index.
Every (sub)array/structure has its own program-location, but the whole arrays-of-arrays variable occupies only one program-index.
Unfortunatly on my current system (and maybe others too) UBO variables are only accessible by program-index from the API.
I tried every combination of names/‘.’/‘[]’ operators and every method I could find to query the program-location of the UBO variables. without succes.

Another thing I noticed while testing some wierd behavior with program-indices (sadly). For further testing I made a UBO’s block array like so:

layout (std140) uniform Block
{
    float array[5];
    vec4 color;
} myUBO[2];

I am able to querry both UBO’s, where “Block” gives the same result as “Block[0]” (might not necessary be a bug, but might explain the next).
When querying the UBO variable indices, I get the following program-indices (example)

|
|
|
|

Nothing wrong so far.
When querying the variable properties (type, size etc.), the variables from “Block[0]” everything works perfectly as expected.
But when querying the variable properties from “Block[1]”, no (valid) result is returned.
For all the variable indices from “Block[1]” it returns the properties of “gl_ModelViewProjectionMatrix”.

After digging into it a litle bit, I thought it would only be possible if you(or the system itself) previously queried properties from that variable,
while the current input is not valid (somehow) and therefore returning the values of the previous querry.
refference: https://www.opengl.org/sdk/docs/man/html/glGetActiveUniform.xhtml

The list of active uniform variables may include both built-in uniform variables (which begin with the prefix “gl_”) as well as user-defined uniform variable names.

This function will return as much information as it can about the specified active uniform variable. If no information is available, length will be 0, and name will be an empty string. This situation could occur if this function is called after a link operation that failed. If an error occurs, the return values length, size, type, and name will be unmodified.

But, right before I qeurry the variables from “Block[1]”, I querried from the variables from “Block[0]”.
So if the last would be the case, I suspected the results of the last variable of “Block[0]”.

When querying the index of each variable seperatly with “GL43.glGetProgramResourceIndex” I could only use (for example)“Block.array”, but not “Block[0].array”.
I have no clue what to think of this, but I keep my drivers responsible for now.

Conclusion
there is no direct way to query a (sub)array length; A possible workaround requires a program-location which are not ussable with UBO-variables;
Its likly that my current system is causing a part of these problems, but it requires further testing.
So for now I believe there is no way to get the results I want.

Thanks for thinking with me!

[Alfonse_Reinheart]

As you mentioned several times, arrays-of-arrays are treated like structures which is true. The problem is that every variable has a program-location and program-index.
Every (sub)array/structure has its own program-location, but the whole arrays-of-arrays variable occupies only one program-index.

It shouldn’t. The specification is very clear about this: arrays of arrays are treated like arrays of structures. Each array element has its own program index. Whether it has a location depends on:

Unfortunatly on my current system (and maybe others too) UBO variables are only accessible by program-index from the API.

Uniform locations are only used for setting the uniform’s value via glUniform calls. So the behavior you’re seeing here is correct because:

I tried every combination of names/’.’/’[]’ operators and every method I could find to query the program-location of the UBO variables. without succes.

Members of uniform blocks don’t have locations. You cannot set their values via glUniform, so there’s no point in assigning them a location.

So OpenGL does not; they get -1.

I am able to querry both UBO’s, where “Block” gives the same result as “Block[0]” (might not necessary be a bug, but might explain the next).

That should not work. Block should not give a legal result from the query function. If an interface block is arrayed, the [] array index is required.

But it is required only when naming the specific block for glGetUniformBlockIndex calls (or the equivalent glGetProgramResourceIndex). I’ll get to that in a bit:

When querying the UBO variable indices, I get the following program-indices (example)

That shouldn’t work. See, the name of the member of an interface block with a instance name is Block.whatever. Even if the block is arrayed, the name of the individual variables in that block are still Block.whatever.

See, even though Block is arrayed, and your shader technically has 2 instances of array and color, the introspection API doesn’t care. If you query the list of uniforms, you will see Block.array and Block.color, without array subscripts.

The reason for that is obvious if you think about it. Because uniforms in a block don’t have individual locations, there is no property that Block[0].array would have which Block[1].array would not. They should return the exact same values for any of the legal introspection queries that the API provides.

So there’s no point in having arrayed blocks generate separate uniforms. So an arrayed interface block only creates one set of variables for that interface, prefixed by the block name without an array index.

When querying the index of each variable seperatly with “GL43.glGetProgramResourceIndex” I could only use (for example)[i]“Block.array”[i], but not “Block[0].array”.

That is exactly how it should be.