Bindless Textures - resident textures Limits

[david_bp]

From Bindless Texture documentation, it is written the following about maximum amount of resident textures:

The amount of storage available for resident images/textures may be less than the total storage for textures that is available. As such, you should attempt to minimize the time a texture spends being resident. Do not attempt to take steps like making textures resident/unresident every frame or something. But if you are finished using a texture for some time, make it unresident.

Is there a way to get this limit for the current hardware? (Ideally through OpenGL API. Or as a last option from hardware vendor specific APIs…)
I didn’t find any other details about this limit than the quote above.

I am working with large texture data, splitted in chunks.
I have 16GB dedicated memory on my GPU.
Making texture resident works well, until I reach around 1.6GB of texture data. From this point, glMakeTextureHandleResidentARB silently fail for the following texture handles:

• no error, but all textures created after that contain only zero values
• and if I call glIsTextureHandleResidentARB after I made the texture resident, it returns false

1.6GB over 16GB seems really low to me as a ratio of available memory for resident textures. I was expecting it to be close to the overall memory space.
But I don’t see what else could make this scenario fail / what other limit I might miss. Any clue is welcome.
When not using bindless textures, the workflow run just fine, regardless of the amount of textures (as long at it fits into GPU memory). And when using fewer texutres, bindless textures work just fine as well.

Thanks in advance.

[Dark_Photon]

@david_bp

From Bindless Texture documentation, it is written the following about maximum amount of resident textures:

The amount of storage available for resident images/textures may be less than the total storage for textures that is available. …

Is there a way to get this limit for the current hardware? …
I didn’t find any other details about this limit than the quote above. …

I have 16GB dedicated memory on my GPU.
Making texture resident works well, until I reach around 1.6GB of texture data.
From this point, glMakeTextureHandleResidentARB silently fail for the following texture handles:

• no error, but all textures created after that contain only zero values
• and if I call glIsTextureHandleResidentARB after I made the texture resident, it returns false
  …
  I don’t see what else could make this scenario fail / what other limit I might miss.

Your second result sound familiar

Are you on Windows?

The limit you’re likely hitting is not the total amount of resident storage (particularly given your GPU mem stats), but instead the maximum number of graphics driver allocations of GPU memory (…if you’re on Windows). This applies to not just bindless textures but across all driver allocations.

On Windows, the limited imposed by the Microsoft WDDM on your graphics driver is 4096 maximum allocations. On Linux, this limit is 4 billion. You can query this limit under Vulkan as VkPhysicalDeviceLimits::maxMemoryAllocationCount. See:

• maxMemoryAllocationCount - Vulkan Hardware Database by Sascha Willems

I don’t know how to query it in GL, but it’s less useful there because you really don’t know what physical GPU allocations your graphics driver is making behind-the-scenes anyway (e.g. streaming buffers for client arrays emulation, ghosted textures to avoid synchronization on updates, texture transfer buffers, etc.)

When not using bindless textures, the workflow run just fine, regardless of the amount of textures (as long at it fits into GPU memory). And when using fewer texutres, bindless textures work just fine as well.

You should be able to get around this limit by pooling your textures into 2D texture arrays. This should consolidate multiple physical texture block allocations into a single texture block allocation (or at least N, where N is the number of texture levels).

On that note, Texture Views may be useful here to make portions of a 2D texture array appear as separate 2D textures. However, I don’t know for sure how this interacts with physical device allocations (maxMemoryAllocationLimit). My “hope” would be that these don’t cost you any extra physical device allocations, …but I don’t know for sure. If this’d be useful to you, you could try it and see.

Oh also… Are you checking for GL errors? If you’re running on an NVIDIA GPU driver, and you have a GL debug message callback plugged in, then when you hit the limit in your app, you may get detailed GL error message info callback from the driver with more details on what limit you hit. For instance, I’ve seen these passed back from NVIDIA, possibly for hitting a limit on the max buffers or textures resident:

GL_OUT_OF_MEMORY error generated. Exceeded buffer residency limits - buffer count
GL_OUT_OF_MEMORY error generated. Failed to allocate memory for texture.

[david_bp]

Thanks for your detailed answer. You raised interesting/good points.

I am indeed on Windows. And with an nVidia GPU.
I queried maxMemoryAllocationCount via Vulkan API on my machine, it’s indeed 4096.

Enabling the GL debug message callback helped getting more detailed error messages. Here is what I am getting:

Unknown internal debug message. The NVIDIA OpenGL driver has encountered an out of memory error. This application might behave inconsistently and fail.

GL_OUT_OF_MEMORY error generated. Exceeded buffer residency limits - buffer count.

Unknown internal debug message. The NVIDIA OpenGL driver detected overflow of memory residency limits. Performance impacts may occur.

Are there error messages you would expect from reaching the maxMemoryAllocationCount limit?
It describes error related to residency limits, but it also specify that it is a “buffer count” limit. The only limit I saw in bindless texture documentation was about an amount of storage, and not a number of buffers.

There are a few points on which I don’t see a clear correlation with the 4096 maxMemoryAllocationCount limit :
First, I don’t get the error message in the 4096 textures range:

• testing with 1MB textures: I could create around 5’600 textures before any error
• testing with 4MB textures: I could create around 2’700 textures before any error
  In both cases, the error doesn’t occur when creating the textures, but later when trying to make these textures resident, and when reaching 1639 simultaneous resident textures.
  So it still makes me wonder if there would be a specific resident textures count limit… (a different limit than maxMemoryAllocationCount. Or a correlated limit).

Also, I am wondering why this limit suddently become visible when enabling blindless textures (as everything was working otherwise, even with way more than 4096 textures). I suppose that handling that many resident textures myself (via glMakeTextureHandleResidentARB) prevent OpenGL from internal creating less than 4096 memory allocations… But it’s just a guess. If you have a better understanding of it, I would be curious to hear about it.

I am working with 3D Textures, so texture arrays and texture views will not help in my case, but thanks for the suggestion. Following/adapting your approach, I am seeing the following way around: group multiple textures together (create fewer but larger 3D textures, and use coord offsets to target “sub-textures”). I did a quick attempt in this direction, when creating 10x larger textures (and 10x less textures), the error message didn’t occur.

What I am still missing at the moment (besides a proper understanding of the situation) is a way to query/estimate the limit causing the issue.
I can indeed not rely on maxMemoryAllocationCount to determine the maximum amount of simultaneous resident textures. Does that resident textures limit (of 1639 in my case) evoke something to you / to anyone?

[Alfonse_Reinheart]

@david_bp

I queried maxMemoryAllocationCount via Vulkan API on my machine, it’s indeed 4096.

It should be noted that this will always be true on Windows, for any implementation, as this is a WDDM-imposed limitation on GPU allocations. It’s there to prevent pathological behavior from applications.

You seem to have compiled evidence that this limitation is not specifically your problem.

[david_bp]

Further notes on the topic, from ARB_bindless_texture.txt:

There may still be indirect limits based on the total amount of storage
available:
[…]

• Limits on the total amount of addressable texture memory imply a
  limit on the total number of resident texture handles.

• Implementations may have (high) limits on the total number of
  texture handles available in the system.

It doesn’t help finding a solution to query the limit, but it highlights that resident textures / texture handles can have (high) limits.

[Dark_Photon]

Well, the GL driver is a deep abstraction, so we don’t really know what the NV driver is doing under-the-covers. But it does seem that bindless textures and bindless buffers hit some of the internal limits where ordinary textures and buffers don’t.

Here are a few random ideas for you to consider.

@david_bp

There are a few points on which I don’t see a clear correlation with the 4096 maxMemoryAllocationCount limit :
First, I don’t get the error message in the 4096 textures range:

• testing with 1MB textures: I could create around 5’600 textures before any error
• testing with 4MB textures: I could create around 2’700 textures before any error

In both cases, the error doesn’t occur when creating the textures, but later when trying to make these textures resident, and when reaching 1639 simultaneous resident textures.

The latter makes sense. Since the dawn of time, GL drivers have been lazy about resource creation. Just because you requested a texture to be created and provided texel data to upload to it didn’t get it allocated on the GPU. In practice, this didn’t occur until the first draw call that demanded that this texture physically exist in order to perform its function. You had to pre-render with the texture to force it to be allocated and populated in GPU memory. Nowadays, making textures resident seems to provide an end-run around this to some degree.

Also, I am wondering why this limit suddently become visible when enabling blindless textures (as everything was working otherwise, even with way more than 4096 textures).

This also makes some sense. Ordinarily, the driver can and does shuffle textures and buffer objects on/off the GPU based on application usage to conform to internal limits. But when you lock a resource down to be resident and/or have a specific address, that serves to tie the drivers hands somewhat. In fact, the bindless specs make a point to mention limiting what you make resident to what you plan to actually use, rather than just making everything resident.

Also, from the app perspective, it’s not obvious if/how the driver pools handle and/or resource allocations across multiple discrete device memory allocations via WDDM. That makes inference difficult.

You might be able to get more detailed info on graphics driver memory allocation and pooling via NV_query_resource and NV_query_resource_tag, or some NVIDIA-specific API (e.g. NVAPI call). Something you could look into if you’re interested.

What I am still missing at the moment (besides a proper understanding of the situation) is a way to query/estimate the limit causing the issue.

I’m interested in that as well. I don’t know exactly what’s going on here either. I’ve just hit this same error with similar GL usage, and while I don’t know, I speculate that the cause is some function of maxMemoryAllocationCount. While it’s not proof, I haven’t seen anyone under Linux complaining about these NV driver errors. I also hit this error when the number of resident bindless textures is just under 4000 (…suspiciously close to the the 4096 Windows limit of maxMemoryAllocationCount ).

[gpu22]

@Dark_Photon

the maximum number of graphics driver allocations of GPU memory (…if you’re on Windows).

how about this MAX number in Linux, esp. Ubuntu? all Linux 4 billion?
and how about it in: all in the main process; all in another thread which create and delete in Loop; all in another process which create and delete in Loop; ?
Thanks!

[gpu22]

@Dark_Photon

I don’t know how to query it in GL, but it’s less useful there because you really don’t know what physical GPU allocations your graphics driver is making behind-the-scenes anyway (e.g. streaming buffers for client arrays emulation, ghosted textures to avoid synchronization on updates, texture transfer buffers, etc.)

OpenGL on Linux(Ubuntu), if there is no way to query or even re-set the MAX number limit.
is there any way to avoid this when running as a server which want to keep running as long as possible.
it seems that kill the process is a method even that is not a good option.
(1)when just running as one main process, it will reach the Limit;
(2)when wrap the Window and Render related res into a thread which is create and delete each loop, also reach the Limit BUT loops count larger than all in one main process;
(3)when wrap them into anther process which is create and delete each loop, it seems that it will be ok to run forever which might because this Limit is for each process? OR for each OpenGL context? THEN is there anyway to ‘clear’&reset the allocation count back to 0, in one main process or in the thread test? SO it could be done at the end of each loop or at intervals？
Thanks a lot!

[gpu22]

@david_bp

I queried maxMemoryAllocationCount via Vulkan API on my machine, it’s indeed 4096.

does maxMemoryAllocationCount return by Vulkan API also means that’s the maxCount for OpenGL? on Linux. Thanks!

[Alfonse_Reinheart]

OpenGL hides all of the information that would make this limit meaningful. The Vulkan maxMemoryAllocationCount is the number of distinct allocations from GPU storage. This is useful because you are the one making allocations directly.

But in OpenGL there’s no way to know if two different textures come from two different allocations. The driver can (and does) have multiple textures reside in the same memory allocation. It can move textures around at more or less any point in time.

[gpu22]

@Dark_Photon

On Windows, the limited imposed by the Microsoft WDDM on your graphics driver is 4096 maximum allocations. On Linux, this limit is 4 billion. You can query this limit under Vulkan as VkPhysicalDeviceLimits::maxMemoryAllocationCount . See:

• maxMemoryAllocationCount - Vulkan Hardware Database by Sascha Willems

maxMemoryAllocationCount means the maxCount that simultaneously exist BUT not the totoal allocated count in Vulkan.
and how about it in OpenGL or Vulkan? on Windows or Ubuntu?
graphics programming - Why do GPUs have limited amount of allocations? - Game Development Stack Exchange

[Alfonse_Reinheart]

@gpu22

maxMemoryAllocationCount means the maxCount that simultaneously exist BUT not the totoal allocated count in Vulkan.

… and? Did he say something different?

There’s not a limit on the number of allocations you can ever do, even if you deallocate them.

@gpu22

and how about it in OpenGL or Vulkan? on Windows or Ubuntu?

As previously stated, such a limit would be meaningless in OpenGL. And the limit is already exposed in Vulkan. So what are you asking about?

Also, just to be clear:

@gpu22

(2)when wrap the Window and Render related res into a thread which is create and delete each loop, also reach the Limit BUT loops count larger than all in one main process;
(3)when wrap them into anther process which is create and delete each loop, it seems that it will be ok to run forever which might because this Limit is for each process? OR for each OpenGL context?

You are not guaranteed that any allocation will ever succeed. There is no guarantee that any implementation will give you that any function that allocates memory will work regardless of what you have done before.

If creating and destroying memory in a loop in perpetuity causes an OOM, the reason for that OOM will not be communicated to you. It’s not something you can know or discover beforehand. There is no “the limit” which you have violated; there’s just the implementation doing what it wants. Whether it “should” succeed or not is irrelevant; if you get an OOM, then you get an OOM and there’s not necessarily anything you could have done to stop it.

It just happens. There’s no limit you violated. There’s nothing you could have known beforehand to prevent it. The only thing you can do is tear down your application and try to restart.

All you can do is avoid engaging in pathological allocation behavior. Don’t make tiny allocations and deallocations. Allocate large blocks of memory at once where possible, and keep them around and reuse them where fesible. Don’t keep textures resident forever. Etc.

[gpu22]

thanks a lot !
Crash after Loop about 70000 times of create, delete, recreate Window on Linux…GL_OUT_OF_MEMORY; Segmentation fault - OpenGL / OpenGL: Advanced Coding - Khronos Forums
would you please take a look at this post?