Most efficient way to use multiple textures with instancing

In my 3D game i am planning to compose every object from many seperate meshes. Every human for example will have a skeleton and every joint will have a mesh. This way i could build a human only using one small cube mesh. I would upload all the matrix data for each cube instance to the shader and then draw all cubes with one draw call. The plan is to be able to draw basically one third of my whole 3D world with just one draw call.

But the problem is with the textures. I obviously dont want all the cubes using the same texture, but i cant change the data i send to the shaders during one draw call. So i would need some kind of set of textures and then i could pass an index value for each instance inside the array of instance datas. What is the most efficient way to do this? I am talking about the best balance between high performance and usability.

This is what i know about the options i have:

  • Texture atlas: stores multiple images inside of one large texture and using the uv-Coordinates i can choose an area on the atlas i would like to use.
    Pros: easy to implement, best performance(?)
    Cons: annoying to set up uv coordinates for different atlasses

  • Texture Array: an array of seperate textures which can be indexed in the shader.
    Pros: good usability
    Cons: is not supported on some platforms

  • 3D Texture: basically a stack of 2D textures and using the depth coordinate you can choose which slice you want to use as a texture
    Pros: good performance, easy to use
    Cons: All textures have to be the same size

I am thinking a 3D texture is what i want to go for and i have already tried to implement it, but ive had troubles with adding the Data of 2D textures together to make one 3D texture, and the indexing fragment shader causes a crash of my graphics drivers:

#version 430

uniform sampler2D sampler[10];



in Vertex{
	vec4 rawPosition;
	vec2 uv;
	vec4 normal;
}vertexIn;

in InstanceData{
	unsigned int textureUnitIndex;
}instance;

out vec4 color;

void main(){
	
	
	unsigned int index = instance.textureUnitIndex;

	color = texture(sampler[index], vertexIn.uv); // CAUSES CRASH

Just to make this complete, this is how i loaded the 3D texture:



//BEFORE THIS I ADDED ALL FILE PATHS OF THE TEXTURES I WANT TO LOAD TO vector<Texture>internalTextures

void TextureSet::loadTextures()
{

	std::vector<unsigned char*> setData; //FINAL DATA OF THE TEXTURE SET

	for (unsigned int i = 0; i < internalTextures.size(); ++i) {

//LOAD ALL PREVIOUSLY ADDED TEXTURES USING SOIL LIBRARY
	internalTextures[i].texData = SOIL_load_image(internalTextures[i].texturePath.c_str(), &internalTextures[i].width, &internalTextures[i].height, &internalTextures[i].channels, SOIL_LOAD_RGBA);

//MAKE SURE TO SAVE THE LARGEST SIZE OUT OF ALL INTERNAL TEXTURES
		if (internalTextures[i].width > width || internalTextures[i].height > height) {
			width = internalTextures[i].width;
			height = internalTextures[i].height;
		}
		
		setData.push_back(internalTextures[i].texData);
		

		
	}

	glCreateTextures(GL_TEXTURE_3D, 1, &setID);
	glActiveTexture(GL_TEXTURE0 + setID-1);
	glBindTexture(GL_TEXTURE_3D, setID);
	glTexParameteri(GL_TEXTURE_3D, GL_TEXTURE_WRAP_S, GL_MIRRORED_REPEAT);
	glTexParameteri(GL_TEXTURE_3D, GL_TEXTURE_WRAP_T, GL_MIRRORED_REPEAT);
	glTexParameteri(GL_TEXTURE_3D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
	glTexParameteri(GL_TEXTURE_3D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
	glTexImage3D(GL_TEXTURE_3D, 0, GL_RGBA, width, height, internalTextures.size(), GL_FALSE, GL_RGBA, GL_UNSIGNED_BYTE, &setData[0]);
	//glTexImage3D(GL_TEXTURE_3D, 0, GL_RGBA, width, height, GL_FALSE, GL_RGBA, GL_UNSIGNED_BYTE, texData);
	glGenerateMipmap(GL_TEXTURE_3D);
	glBindTexture(GL_TEXTURE_3D, 0);

//FREE ALL THE POINTERS WITH THE TEXTURE DATA
for (unsigned int i = 0; i < internalTextures.size(); ++i) {
SOIL_free_image_data(internalTextures[i].texData);
}
}

Is there a more efficient way than 3D textures or is there something i got wrong about the options?
how do i correctly assemble and index a 3D texture?

[QUOTE=stimulate;1284722]This is what i know about the options i have:

  • Texture atlas: stores multiple images inside of one large texture and using the uv-Coordinates i can choose an area on the atlas i would like to use.
    Pros: easy to implement, best performance(?)
    Cons: annoying to set up uv coordinates for different atlasses

  • Texture Array: an array of seperate textures which can be indexed in the shader.
    Pros: good usability
    Cons: is not supported on some platforms

  • 3D Texture: basically a stack of 2D textures and using the depth coordinate you can choose which slice you want to use as a texture
    Pros: good performance, easy to use
    Cons: All textures have to be the same size[/QUOTE]

That’s a good starter list. A few additions to them:

  1. Texture atlas con: unwanted cross-image filtering can be a real pain to avoid, particularly if you use MIPmaps.
  2. Texture atlas: your pro is TBD based on your image size, the max supported GL 2D texture size, and how many textures you need to access at once.
  3. 3D Texture con: Your con also applies to texture arrays (at least within a single GL texture). Internal format must also be the same in both cases.

Are you sure you need to support GPUs/drivers that don’t support texture arrays?

Also add to this Bindless Textures. However, if you need to support GPU drivers w/o texture arrays, I suspect you’ll be less than satisfied with the list that supports bindless textures. But it’s worth checking on that.

Texture atlas:
Con: “bleeding” between tiles, can’t use it for tiles that need to wrap.
Pro: works with any version.

Texture array - are you talking about a uniform array of 2D textures (i.e. sampler2D[]) or an array texture (sampler2DArray)?

Uniform array:
Con: needs one texture unit per texture, can only be indexed with a dynamically-uniform expression (or only with a uniform expression in older versions).
Pro: each texture is a separate texture with its own dimensions, format and parameters.

Array texture:
Pro: only needs one texture unit, can be accessed with arbitrary expressions, can store a large amount of texture data.
Con: layers aren’t separate textures, so they must all have the same dimensions, format and parameters.

3D Texture:
Pro: requires a lower OpenGL version than a 2D array texture.
Con: mipmapping operates on all 3 axes, so using mipmaps will blend between slices, effectively precluding the use of mipmaps if you’re using a 3D texture as a “ghetto” 2D array texture.

i personally favour array textures, there is only 1 obvious “problem”: constant resolution for all layers
1 way to get around that problem is re-texturing of all models you use, and make them fit into (let’s say) 1024 x 1024 RGBA
(that would be ~4MB per texture layer)

with that appproach, you can even switch textures within 1 mesh (instance):
–> stream a “int mapKd” as vertex attribute to the vertex shader, pass it to the fragment shader and access your array texture with that index

i draw all of my meshes as GL_TRIANGLES, the vertices look like this:
struct Vertex {
vec3 Position;
vec2 TexCoord;
vec3 Normal;
int MaterialIndex;
};

i’m using “Material” structs in my fragmentshader, all materials (put into a buffer) are bound to an uniform block
materials look like that: (GLSL)

struct Material {
    vec4 Ka, Kd, Ks;
    float Ns;
    float d;
    int map_Kd;
};

layout (std140, binding = 1) uniform MaterialBlock
{
    Material Materials[MAX_MATERIALS];
};

uniform sampler2DArray textureKd;

void main()
{
// ...

int map_Kd = Materials[vs_out.materialindex].map_Kd;
vec3 Kd = Materials[vs_out.materialindex].Kd.rgb * texture(textureKd, vec3(vs_out.texcoord, map_Kd)).rgb;
}

the streamed vertex attribute “int MaterialIndex” is used to access the correct material
that material itself has then the texture index “int map_Kd” which is use to access the correct texture (diffuse)
that way i can enable / disable textures within 1 mesh, just by avoiding a uniform “switch ON/OFF” variable
if a certain face doesnt need textures at all, the texture index will be 0 and the first layer of the tarray texture is completely white

if you want to use different textures for the same instanced rendered mesh, stream another instanced “int override_mapKd” to the vertex shader and pass it instead of the “int MaterialIndex”

RESTRICTIONS:
– of course all textures have the same resolution
– maximum number of materials per drawcall is limited (GL_MAX_UNIFORM_BLOCK_SIZE bytes per block)
(but i think openGl requires at least 16KB, that is ~250 different materials per drawcall)
(my [old] NVIDIA GT 640 has 65536 bytes, that means about 1000 different materials / drawcall)

You guys convinced me to use texture arrays, because it seemed like they were created just for this purpose and will therefore be faster than i expected.
Anyways, i am currently having a weird issue. When i am indexing the texture2DArray through the third coordinate in my texture coordinate vector, all i can get to show is either the zeroth or the last element in the texture array. I have 5 textures in my texture array and when i check the textureIndex variable in the InstanceData (when i am uploading it to the openGL buffer) it has the value i expected it to. When the index variable is 0 it shows the first texture (of course) but when the variable is 1, 2, 3 or 4 it always shows the last texture. When i hardcode the index inside of the fragment shader to say 2, it renders the third texture just how i want it to.
What could this be? i checked and i am doing nothing inside of the other shader stages exept passing it to the next stage.
Fragment Shader:

#version 430
#define PI 3.1415926535897932384626433832795
uniform vec3 lightColor;

uniform sampler2DArray texSampler;

uniform vec3 ambientLight;
uniform vec3 lightPosition;
uniform vec3 cameraPosition;


in Vertex{
	vec4 rawPosition;
	vec2 uv;
	vec4 normal;
}vertexIn;

in InstanceData{
	unsigned int textureUnitIndex;
}instance;

out vec4 color;

void main(){
	
	vec3 normal = normalize(vec3(vertexIn.normal.x, vertexIn.normal.y, vertexIn.normal.z));
	vec3 worldPosition = vec3(vertexIn.rawPosition.x, vertexIn.rawPosition.y, vertexIn.rawPosition.z);
	
	vec3 lightVector = normalize(lightPosition - worldPosition);
	vec3 cameraVector = normalize(cameraPosition - worldPosition);
	vec3 halfVec = normalize(lightVector + cameraVector);

	float lightAngle = clamp(dot(lightVector, normal), 0.0f, 1.0f);
	float specAngle = clamp(dot(halfVec, normal), 0.0f, PI);
	float specular = clamp(pow(specAngle, 128), 0.0f, 1.0f);

////
	unsigned int index = instance.textureUnitIndex;

	vec3 textureCoordinates = vec3(vertexIn.uv.x, vertexIn.uv.y, index);
	vec4 actColor =  texture(texSampler, textureCoordinates);
/////

	vec4 diffuseLight = sqrt(actColor * vec4(lightColor, 1.0)) * lightAngle;
	vec4 specularLight = 1.0f - (cos(vec4(lightColor, 1.0f) * specular));

	color = actColor * (vec4(ambientLight, 1.0) + diffuseLight) + specularLight;
}

Other than that textureArrays were much easier to implement than expected.

  1. there is no “unsigned int” variable type in GLSL, instead there is “uint”
  2. variables passed from vertex shader to fragment shader will be (by default) interpolated

consider a rectangle: your vertex shader gets invoked 4 times, once per corner
each corner has texture coordinates that are either 0 or 1, but within the rendered rectangle you got the full texture colors rendered, not only the colors of the corner points of the texture (i mean the very last texels in the texture)

thats because you pass:

[NO_QUALIFIER] out vec2 texcoord;

[NO_QUALIFIER] means “smooth”
https://www.opengl.org/wiki/Type_Qualifier_(GLSL)#Interpolation_qualifiers

but you cant interpolate integer-type variables! (at least not in a way that makes sense)
instead you have to pass “int” or “uint” variables with a qualifier “flat”
“flat” means there will not be any interpolation, just the last processed vertex will pass the variable to the fragment shader stage
that means all the fragments covered by the primitive will have the same (latest vertex’s) variable
example: a triangle gets renderd, vertex 0 writes the “flat” variable, then vertex 1 overrides it and at finally vertex 2 overrides it again, making the previous 2 written values obsolete

didnt you check your shaders for compilation / linking errors ?
https://www.opengl.org/wiki/Shader_Compilation#Example

try that:

in InstanceData{
	flat uint textureUnitIndex;
}instance;

of course that code has to match the block in the vertex shader

Unfortunately this did not change anything :confused:
but textureUnitIndex is not a Vertex attribute anyways (right?). its just data i pass independently from the vertecies. i specified to advance in the buffer after every instance using glVertexArrayBindingDivisor().
I have no shader compiling/linking errors.

This is how i initialize my VAO with the (at this point empty) instance data buffer

glCreateVertexArrays(1, &MeshData::static_vaoID);
		glCreateBuffers(1, &MeshData::static_vboID);
		glCreateBuffers(1, &MeshData::static_iboID);
		glCreateBuffers(1, &MeshData::static_instanceDataID);

		glEnableVertexArrayAttrib(MeshData::static_vaoID, 0);
		glEnableVertexArrayAttrib(MeshData::static_vaoID, 1);
		glEnableVertexArrayAttrib(MeshData::static_vaoID, 2);
		glEnableVertexArrayAttrib(MeshData::static_vaoID, 3);
		glEnableVertexArrayAttrib(MeshData::static_vaoID, 4);
		glEnableVertexArrayAttrib(MeshData::static_vaoID, 5);
		glEnableVertexArrayAttrib(MeshData::static_vaoID, 6);
		glEnableVertexArrayAttrib(MeshData::static_vaoID, 7);


		//associate vertexArray with vboID and indexArray with iboID
		glNamedBufferStorage(MeshData::static_vboID, sizeof(Vertex)*MeshData::totalVertexNum, &MeshData::allVertices[0], 0);

		glNamedBufferStorage(MeshData::static_iboID, sizeof(unsigned int)*MeshData::totalIndexNum, &MeshData::allIndices[0], 0);
		glVertexArrayElementBuffer(MeshData::static_vaoID, MeshData::static_iboID);

		//vaoID vertex attrib array binding location 0 -> vboID
		glVertexArrayVertexBuffer(MeshData::static_vaoID, 0, MeshData::static_vboID, 0, sizeof(Vertex));

		
                 //Vertex Attributes
		glVertexArrayAttribBinding(MeshData::static_vaoID, 0, 0);
		glVertexArrayAttribFormat(MeshData::static_vaoID, 0, 3, GL_FLOAT, GL_FALSE, offsetof(Vertex, position));
		glVertexArrayAttribBinding(MeshData::static_vaoID, 1, 0);
		glVertexArrayAttribFormat(MeshData::static_vaoID, 1, 2, GL_FLOAT, GL_FALSE, offsetof(Vertex, uv));
		glVertexArrayAttribBinding(MeshData::static_vaoID, 2, 0);
		glVertexArrayAttribFormat(MeshData::static_vaoID, 2, 3, GL_FLOAT, GL_FALSE, offsetof(Vertex, normal));


                // Instance Data Buffer
		glNamedBufferStorage(MeshData::static_instanceDataID, sizeof(InstanceData)*MAX_INSTANCES, nullptr, GL_DYNAMIC_STORAGE_BIT);

		glVertexArrayAttribBinding(MeshData::static_vaoID, 3, 1);
		glVertexArrayAttribFormat(MeshData::static_vaoID, 3, 4, GL_FLOAT, GL_FALSE, offsetof(InstanceData, meshMatrix));

		glVertexArrayAttribBinding(MeshData::static_vaoID, 4, 1);
		glVertexArrayAttribFormat(MeshData::static_vaoID, 4, 4, GL_FLOAT, GL_FALSE, offsetof(InstanceData, meshMatrix) + sizeof(float) * 4);

		glVertexArrayAttribBinding(MeshData::static_vaoID, 5, 1);
		glVertexArrayAttribFormat(MeshData::static_vaoID, 5, 4, GL_FLOAT, GL_FALSE, offsetof(InstanceData, meshMatrix) + sizeof(float) * 8);

		glVertexArrayAttribBinding(MeshData::static_vaoID, 6, 1);
		glVertexArrayAttribFormat(MeshData::static_vaoID, 6, 4, GL_FLOAT, GL_FALSE, offsetof(InstanceData, meshMatrix) + sizeof(float) * 12);

		glVertexArrayAttribBinding(MeshData::static_vaoID, 7, 1);
		glVertexArrayAttribFormat(MeshData::static_vaoID, 7, 1, GL_UNSIGNED_INT, GL_FALSE, offsetof(InstanceData, textureSetIndex));


		glVertexArrayVertexBuffer(MeshData::static_vaoID, 1, MeshData::static_instanceDataID, 0, sizeof(InstanceData));
		glVertexArrayBindingDivisor(MeshData::static_vaoID, 1, 1);

And this is how i render

if (meshData.instanceVector.size() > 0) {

                        //here i update the Instance Data Buffer. this works fine, at least for the matrices. textureUnitIndex is what its supposed to be at this point
			glNamedBufferSubData(MeshData::static_instanceDataID, 0, sizeof(InstanceData)*(meshData.instanceVector.size()), &meshData.instanceVector[0]);

			currentShader.addUniform("texSampler", (int)meshData.textureSet);
			glActiveTexture(GL_TEXTURE0 + meshData.textureSet);
			glBindTexture(GL_TEXTURE_2D_ARRAY, meshData.textureSet);
			//
			glDrawElementsInstanced(GL_TRIANGLES, meshData.indexCount, GL_UNSIGNED_INT, (void*)(sizeof(unsigned int)*(meshData.indexOffset)), meshData.instanceVector.size());
			//
			glBindTexture(GL_TEXTURE_2D_ARRAY, 0);
			glActiveTexture(GL_TEXTURE0);

			if (renderNormals) {
				useGLSLProgram("NormalShader");
				updateShader();
				glDrawElementsInstanced(GL_TRIANGLES, meshData.indexCount, GL_UNSIGNED_INT, (void*)(sizeof(unsigned int)*(meshData.indexOffset)), meshData.instanceVector.size());
			}
		}
		glInvalidateBufferData(MeshData::static_instanceDataID);
		glNamedBufferSubData(MeshData::static_instanceDataID, 0, sizeof(InstanceData)*(meshData.instanceVector.size()), nullptr);

try to figure it out step by step:
you said setting the index value directly (hardcoded) into the fragment shader works
now try to pass an arbitrary value from your verex shader to your fragment shader, like this:

vertex shader source:

out VS_FS {
smooth vec2 texcoord;
flat uint textureindex;
}vs_out;

void main()
{
gl_Position = vec4...

// pass a value from the vertex shader
vs_out.textureindex = 2;
vs_out.texcoord= ...;// vertex attribute

}

fragment shader source:

in VS_FS {
smooth vec2 texcoord;
flat uint textureindex;
}vs_out;

void main()
{
vec4 texel = texture(mytexarr, vec3(vs_out.texcoord, vs_out.textureindex));
// write it to the output variable
}

does that work ?
if so, go a step back and pass a uint vertex attribute to “flat uint textureindex;”

if that doesnt work, check if your vertex array actually streams “integer-type” variables instead of floating point variables
use glVertexAttribIPointer(…) to do that

it really seems to be something with how the data is uploaded to the buffer… when i hardcode it into the vertex shader it works fine too…
i will keep looking some more though, good tips here

so it turns out you need to use glVertexAttribIFormat() instead of glVertexAttribFormat for some reason. Wow. Im so glad this texture stuff is through… thank you all so much :slight_smile:

glVertexAttribFormat() always generates floats. It can’t tell that the GLSL variable will be an integer (the shader may not be bound or even exist when the attribute arrays are set up).

thanks for the clarification. Knowing that stuff like this matters will definitely help me with later shading and buffer uploads.