Bindless Textures - resident textures Limits

From Bindless Texture documentation, it is written the following about maximum amount of resident textures:

The amount of storage available for resident images/textures may be less than the total storage for textures that is available. As such, you should attempt to minimize the time a texture spends being resident. Do not attempt to take steps like making textures resident/unresident every frame or something. But if you are finished using a texture for some time, make it unresident.

Is there a way to get this limit for the current hardware? (Ideally through OpenGL API. Or as a last option from hardware vendor specific APIs…)
I didn’t find any other details about this limit than the quote above.

I am working with large texture data, splitted in chunks.
I have 16GB dedicated memory on my GPU.
Making texture resident works well, until I reach around 1.6GB of texture data. From this point, glMakeTextureHandleResidentARB silently fail for the following texture handles:

  • no error, but all textures created after that contain only zero values
  • and if I call glIsTextureHandleResidentARB after I made the texture resident, it returns false

1.6GB over 16GB seems really low to me as a ratio of available memory for resident textures. I was expecting it to be close to the overall memory space.
But I don’t see what else could make this scenario fail / what other limit I might miss. Any clue is welcome.
When not using bindless textures, the workflow run just fine, regardless of the amount of textures (as long at it fits into GPU memory). And when using fewer texutres, bindless textures work just fine as well.

Thanks in advance.

Your second result sound familiar :slight_smile:

Are you on Windows?

The limit you’re likely hitting is not the total amount of resident storage (particularly given your GPU mem stats), but instead the maximum number of graphics driver allocations of GPU memory (…if you’re on Windows). This applies to not just bindless textures but across all driver allocations.

On Windows, the limited imposed by the Microsoft WDDM on your graphics driver is 4096 maximum allocations. On Linux, this limit is 4 billion. You can query this limit under Vulkan as VkPhysicalDeviceLimits::maxMemoryAllocationCount. See:

I don’t know how to query it in GL, but it’s less useful there because you really don’t know what physical GPU allocations your graphics driver is making behind-the-scenes anyway (e.g. streaming buffers for client arrays emulation, ghosted textures to avoid synchronization on updates, texture transfer buffers, etc.)

You should be able to get around this limit by pooling your textures into 2D texture arrays. This should consolidate multiple physical texture block allocations into a single texture block allocation (or at least N, where N is the number of texture levels).

On that note, Texture Views may be useful here to make portions of a 2D texture array appear as separate 2D textures. However, I don’t know for sure how this interacts with physical device allocations (maxMemoryAllocationLimit). My “hope” would be that these don’t cost you any extra physical device allocations, …but I don’t know for sure. If this’d be useful to you, you could try it and see.

Oh also… Are you checking for GL errors? If you’re running on an NVIDIA GPU driver, and you have a GL debug message callback plugged in, then when you hit the limit in your app, you may get detailed GL error message info callback from the driver with more details on what limit you hit. For instance, I’ve seen these passed back from NVIDIA, possibly for hitting a limit on the max buffers or textures resident:

GL_OUT_OF_MEMORY error generated. Exceeded buffer residency limits - buffer count
GL_OUT_OF_MEMORY error generated. Failed to allocate memory for texture.

Thanks for your detailed answer. You raised interesting/good points.

I am indeed on Windows. And with an nVidia GPU.
I queried maxMemoryAllocationCount via Vulkan API on my machine, it’s indeed 4096.

Enabling the GL debug message callback helped getting more detailed error messages. Here is what I am getting:

Unknown internal debug message. The NVIDIA OpenGL driver has encountered an out of memory error. This application might behave inconsistently and fail.

GL_OUT_OF_MEMORY error generated. Exceeded buffer residency limits - buffer count.

Unknown internal debug message. The NVIDIA OpenGL driver detected overflow of memory residency limits. Performance impacts may occur.

Are there error messages you would expect from reaching the maxMemoryAllocationCount limit?
It describes error related to residency limits, but it also specify that it is a “buffer count” limit. The only limit I saw in bindless texture documentation was about an amount of storage, and not a number of buffers.

There are a few points on which I don’t see a clear correlation with the 4096 maxMemoryAllocationCount limit :
First, I don’t get the error message in the 4096 textures range:

  • testing with 1MB textures: I could create around 5’600 textures before any error
  • testing with 4MB textures: I could create around 2’700 textures before any error
    In both cases, the error doesn’t occur when creating the textures, but later when trying to make these textures resident, and when reaching 1639 simultaneous resident textures.
    So it still makes me wonder if there would be a specific resident textures count limit… (a different limit than maxMemoryAllocationCount. Or a correlated limit).

Also, I am wondering why this limit suddently become visible when enabling blindless textures (as everything was working otherwise, even with way more than 4096 textures). I suppose that handling that many resident textures myself (via glMakeTextureHandleResidentARB) prevent OpenGL from internal creating less than 4096 memory allocations… But it’s just a guess. If you have a better understanding of it, I would be curious to hear about it.

I am working with 3D Textures, so texture arrays and texture views will not help in my case, but thanks for the suggestion. Following/adapting your approach, I am seeing the following way around: group multiple textures together (create fewer but larger 3D textures, and use coord offsets to target “sub-textures”). I did a quick attempt in this direction, when creating 10x larger textures (and 10x less textures), the error message didn’t occur.

What I am still missing at the moment (besides a proper understanding of the situation) is a way to query/estimate the limit causing the issue.
I can indeed not rely on maxMemoryAllocationCount to determine the maximum amount of simultaneous resident textures. Does that resident textures limit (of 1639 in my case) evoke something to you / to anyone?

It should be noted that this will always be true on Windows, for any implementation, as this is a WDDM-imposed limitation on GPU allocations. It’s there to prevent pathological behavior from applications.

You seem to have compiled evidence that this limitation is not specifically your problem.

Further notes on the topic, from ARB_bindless_texture.txt:

There may still be indirect limits based on the total amount of storage
available:
[…]

  • Limits on the total amount of addressable texture memory imply a
    limit on the total number of resident texture handles.

  • Implementations may have (high) limits on the total number of
    texture handles available in the system.

It doesn’t help finding a solution to query the limit, but it highlights that resident textures / texture handles can have (high) limits.

Well, the GL driver is a deep abstraction, so we don’t really know what the NV driver is doing under-the-covers. But it does seem that bindless textures and bindless buffers hit some of the internal limits where ordinary textures and buffers don’t.

Here are a few random ideas for you to consider.

The latter makes sense. Since the dawn of time, GL drivers have been lazy about resource creation. Just because you requested a texture to be created and provided texel data to upload to it didn’t get it allocated on the GPU. In practice, this didn’t occur until the first draw call that demanded that this texture physically exist in order to perform its function. You had to pre-render with the texture to force it to be allocated and populated in GPU memory. Nowadays, making textures resident seems to provide an end-run around this to some degree.

This also makes some sense. Ordinarily, the driver can and does shuffle textures and buffer objects on/off the GPU based on application usage to conform to internal limits. But when you lock a resource down to be resident and/or have a specific address, that serves to tie the drivers hands somewhat. In fact, the bindless specs make a point to mention limiting what you make resident to what you plan to actually use, rather than just making everything resident.

Also, from the app perspective, it’s not obvious if/how the driver pools handle and/or resource allocations across multiple discrete device memory allocations via WDDM. That makes inference difficult.

You might be able to get more detailed info on graphics driver memory allocation and pooling via NV_query_resource and NV_query_resource_tag, or some NVIDIA-specific API (e.g. NVAPI call). Something you could look into if you’re interested.

I’m interested in that as well. I don’t know exactly what’s going on here either. I’ve just hit this same error with similar GL usage, and while I don’t know, I speculate that the cause is some function of maxMemoryAllocationCount. While it’s not proof, I haven’t seen anyone under Linux complaining about these NV driver errors. I also hit this error when the number of resident bindless textures is just under 4000 (…suspiciously close to the the 4096 Windows limit of maxMemoryAllocationCount :thinking:).