Python - glMapBuffer question

OpenGL
1

I have a Python project which requires a pointer to GPU memory using OpenGL

So far I have attempted the following:

import numpy as np

from OpenGL.GL import *
import cv2
import pygame

height = 480
width = 640

#pygame
#-----------------------

pygame.init()
pygame.display.set_caption('Texture Receiver Example')
pygame.display.set_mode((width, height),
                        pygame.OPENGL | pygame.DOUBLEBUF)

TextureID = glGenTextures(1)

glBindTexture(GL_TEXTURE_2D, TextureID)

#--------------------------------
pbo = glGenBuffers(1) # create a pixel buffer object
openglframe =np.random.randint(low=0,high=255,size=(height,width,4)) # array with the resolution of the image I want to use, in 4 channels for RGBA, used for the byte size of glBufferData

glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo)
glBufferData(GL_PIXEL_PACK_BUFFER,openglframe ,  GL_DYNAMIC_READ)  # configures the pixel buffer

openglcudamatcudamat = cv2.cuda.GpuMat(height,width,cv2.CV_8UC3) # temporary texture used for mapping the address from OpenGL as a CUDA GpuMat
outputcudamat = cv2.cuda.GpuMat(height,width,cv2.CV_8UC3) # the OpenGL texture should end up as this CUDA GpuMat


glActiveTexture(GL_TEXTURE0) # activate the OpenGL texture slot
glBindTexture(GL_TEXTURE_2D, TextureID) # selects the texture
glTexImage2D(GL_TEXTURE_2D, 0, 3, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, openglframe) # Load the image into OpenGL
glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo) #selects the pixel buffer
glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_UNSIGNED_BYTE,0)


# After this is the Python part
mapdata = glMapBuffer(GL_PIXEL_PACK_BUFFER,GL_READ_ONLY) # returns a ctype pointer of the pixel buffer


openglcudamat=cv2.cuda.createGpuMatFromCudaMemory(rows=height,cols=width,type=cv2.CV_8UC4,cudaMemoryAddress=mapdata)
openglcudamat.copyTo(outputcudamat) # copy the OpenGL texture from temporary CUDA GpuMat 

# And finally, unmap the pixel buffer in OpenGl
glUnmapBuffer(GL_PIXEL_PACK_BUFFER)

outputimage = outputcudamat.download()
cv2.imshow("test",outputimage)
print (outputimage)

the line in question is

mapdata = glMapBuffer(GL_PIXEL_PACK_BUFFER,GL_READ_ONLY)

does this return a pointer to system memory, or to GPU memory?

2

The pointer will be a host address space pointer, but it could potentially point to a region of memory that is mapped over the PCIe bus. I don’t think the spec makes specific promises here, implementations (i.e. drivers) are free to place buffer memory where they think is best.

For accessing buffers with both CUDA and OpenGL without copying through host memory, there is the OpenGL Interop API.

There are the Nvidia specifc NV_shader_buffer_{load,store} extensions which give you a “GPU address”, but those addresses are meant to be sent back to a shader for accessing the buffer contents there, so that does not give you something you can do much with on the python side. Perhaps you can describe what you want to do with the pointer to GPU memory?

3

I suspect that you’re going to need to use CUDA to get that pointer.

glMapBuffer will (probably) return a pointer to GPU memory mapped to the CPU’s address space; i.e. it won’t be acceptable to that function as it will be in the CPU’s address space, not the GPU’s. And it’s not even required to be a physical mapping; the implementation is allowed to use a shadow buffer and copy the data when you call glUnmapBuffer.

4

Perhaps you can describe what you want to do with the pointer to GPU memory?

I have a Python project where I need to transfer an OpenGL image to CUDA in OpenCV, and currently I am doing it through system memory, I would like to keep everything on the GPU

The author of the Python implementation of OpenCV has added the cv2.cuda.createGpuMatFromCudaMemory function, and has responded to me on the OpenCV forums, quote:

if you can get a pointer to the GPU memory in the same context everything should work.

5

And it’s not even required to be a physical mapping; the implementation is allowed to use a shadow buffer and copy the data when you call glUnmapBuffer

Here are my findings, if I do this

openglcudamat = cv2.cuda.GpuMat(height,width,cv2.CV_8UC3)
mapdata = glMapBuffer(GL_PIXEL_PACK_BUFFER,GL_READ_ONLY) 
print(mapdata)
print(openglcudamat.cudaPtr())
openglcudamat=cv2.cuda.createGpuMatFromCudaMemory(rows=height,cols=width,type=cv2.CV_8UC4,cudaMemoryAddress=mapdata)
print(openglcudamat.cudaPtr())

the print outputs are:

2638503923712
47485812736
2638503923712

from this it looks like the memory address assignment simply works, but the actual address being assigned to seems quite different, the address produced by glMapBuffer is 2 digits longer than a working CUDA memory address

6

I’m not sure how much that really tells you…

I think the issue is made a little bit more challenging because of the Python and OpenCV wrappers around OpenGL/CUDA objects - those are also the parts I’m least familiar with, so perhaps I’m biased :wink:
I still think the OpenGL interop functionality in CUDA is the way to go:

  • use cudaGraphicsGLRegisterBuffer to get a cudaGraphicsResource that corresponds to your OpenGL buffer object
  • use cudaGraphicsMapResources to map the resource from the previous step for use by CUDA (note that you must not access the buffer from OpenGL until you call the corresponding cudaGraphicsUnmapResources)
  • use cudaGraphicsResourceGetMappedPointer to get a pointer to the memory (CUDA calls this a “device pointer”) and that is what I would expect is fine to pass to cv2.cuda.createGpuMatFromCudaMemory.