Image CPU to GPU Transfer Speeds

Hello, I’m looking to transfer images from CPU to GPU in under 10ms. Currently, it varies between 40ms - 60ms.

The graphics card I’m using is NVIDIA GeForce RTX 3060. Currently, the graphics card is connected to the motherboard using a PCIe Gen2.

The bitmap images are 2040 pixels x 1536 pixels in an RGBA format.

Question 1: What would be an estimate for transferring images from the CPU to GPU using PCIe Gen2, Gen3 and Gen4?

My current understanding is that a Transfer only queue family uses Direct Memory Access (DMA). (Please correct me if I am wrong). I decided to use this to improve the CPU to GPU transfer speed, however it seems to have had no effect.

Question 2: What could be the reason for this?

Question 3: Are there any other ways to improve transfer speeds?

Question 4: Would there be a difference in image transfer rates between Vulkan and other GPU APIs?

pretty archaic

2040 × 1536 × 4 B = 12 MB. A small sample to measure actual bandwith. Possibly includes some notable overhead. The bus is unlikely the bottleneck. Considering the age of your motherboard, one should consider also the RAM bandwith.

Conveniently, bandwith is roughly doubled with each version.

Because the same is used between CPU and GPU. There’s no other connection except the PCIe bus. You are just being explicit the transfer should be asynchronous (which it likely will anyway out of necessity). Considering it is CPU → GPU transfer direction, I think such distinction makes further less sense…

It could make a difference within GPU. Combined queue copy should be faster (and synchronous\competing with shader execution). Dedicated queue copy could be slower but asynchronous.

You could try the BAR memory type. It is intended for these frequent low latency transfers into the GPU. Before PCIe 4, it is limited to 256 MB though (of which some might be further reserved).

In theory, there shouldn’t be.

Some interesting tech like DirectStorage might come to DirectX first. I think something to that effect is exposed in Vulkan now.

VK_EXT_host_image_copy could also be useful to have less overhead with staging and management of the Tiling and Layout.

1 Like