Video stream by GPU with a PBO to load an ImageQT in OpenGL

[sacha]

I plan to develop a tool for realtime video manipulation using C++, QT 4.7.4 and OpenGL and i’m on visual studio 2008. I check many web site but my code doesn’t work since I try with PBO. At the moment I imagine a following sequence of steps:

1. load image file into QTimage
2. copy data into PBO with glmapBuffer
3. transfert into a texture with glTextSubImage2D

Is correct ?

And after succes with several image then I’ll begin with video.

To see if I din’t miss somthing a explaine my code:

INIT I use glut so I initialised with :

glutInit(&iArgc, cppArgv); 
glutInitDisplayMode( GLUT_RGBA | GLUT_DOUBLE  | GLUT_ALPHA);
glutInitWindowSize(1000, 1000);
glutInitWindowPosition(500, 10);
glutCreateWindow("Texture"); 

glEnable(GL_TEXTURE_2D);

In my MAIN:

GLuint pboIds[2];
glGenBuffers(N_MAX_BUFFERS, pboIds);
//for two buffers
glBindBuffer(GL_PIXEL_PACK_BUFFER, pboIds[iBuffer]);
glBufferData(GL_PIXEL_PACK_BUFFER, DATA_SIZE, NULL, GL_STREAM_DRAW);

glBindBuffer(GL_PIXEL_UNPACK_BUFFER, 0);
glReadBuffer(GL_FRONT);

glGenTextures(1,&texture)

In a fonction loadImage I do the step 1:

QImage image;
image.load(name);
imageGL = QGLWidget::convertToGLFormat(image);
imageGL = (image.convertToFormat(QImage::Format_RGB888));//format is important

And the step 2 :

for (int iBuffer = 0; iBuffer < N_MAX_BUFFERS; ++iBuffer)
  {
    glBindBuffer(GL_PIXEL_PACK_BUFFER_EXT, pboIds[iBuffer]);
    glBufferData(GL_PIXEL_PACK_BUFFER_EXT, DATA_SIZE, NULL, GL_STREAM_DRAW);

    void * pPixelsPBO = glMapBuffer(GL_PIXEL_PACK_BUFFER_EXT, GL_WRITE_ONLY);
    // copy original image into the buffer (Step 2)
    if(pPixelsPBO)
    {
      pPixelsPBO = new unsigned char[imageGL.width() * imageGL.height() * 3];
      memcpy(pPixelsPBO, imageGL.bits(), DATA_SIZE);
    }
    glUnmapBuffer(GL_PIXEL_PACK_BUFFER_EXT
    glBindBuffer(GL_PIXEL_PACK_BUFFER_EXT, 0);
  }

And finally in the glutDisplayFunc():

//step 3
glBindTexture(GL_TEXTURE_2D, texture);
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, pboIds[Index]);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, imageGL.width(), imageGL.height(), GL_RGB, GL_UNSIGNED_BYTE, 0);
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, 0);
///

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
glPushMatrix();
glBindTexture(GL_TEXTURE_2D, texture);
glColor4f(1, 1, 1, 1);

glBegin(GL_QUADS);

glTexCoord2i(0,0); glVertex2i(0,imageGL.height());
glTexCoord2i(0,1); glVertex2i(0,0);
glTexCoord2i(1,1); glVertex2i(imageGL.width(),0);
glTexCoord2i(1,0); glVertex2i(imageGL.width(),imageGL.height());
glEnd();

glBindTexture(GL_TEXTURE_2D, 0);
glDisable(GL_TEXTURE_2D);
glPopMatrix();
glutSwapBuffers();

//to load the next data from image 
// I do the step 1 et 2 at the end
LoadImg("./Code.jpg");

I get only an error on glTexSubImage2D() INVALID_OPERATION means generally format problem but I check many different format.

I don’t know if i completly lost or if I close to succes.
should I use VBO and VAO or the ARB fonction ?

[Dark_Photon]

Look at the causes of a GL_INVALID_OPERATION error being triggered by a glTexSubImage2D() call for clues (see the list at the bottom of the linked man page).

So where did you allocate the storage for the texture (glTexImage2D() or glTexStorage2D())?

glTexSubImage2D() only updates the contents of an existing texture MIP, whereas:

• glTexImage2D() allocates the storage and optionally updates the contents, and
• glTexStorage2D() only allocates the storage.

Also…

@sacha

And the step 2 :

...
    void * pPixelsPBO = glMapBuffer(GL_PIXEL_PACK_BUFFER_EXT, GL_WRITE_ONLY);
    // copy original image into the buffer (Step 2)
    if(pPixelsPBO)
    {
      pPixelsPBO = new unsigned char[imageGL.width() * imageGL.height() * 3];
      memcpy(pPixelsPBO, imageGL.bits(), DATA_SIZE);
    }
    glUnmapBuffer(GL_PIXEL_PACK_BUFFER_EXT
...

Take a closer look at what you’re doing here. Presumably you want pPixelsPBO to point to the mapped buffer, not a malloced block of heap memory.

[sacha]

I did it in my LoadImage() fonction but I read is better before glGenBuffers()
So now it’s in my MAIN after the initialisation and before glGenBuffers()

glGenTextures(1,&texture);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP);
glTexImage2D(GL_TEXTURE_2D, 0, **GL_SRGB8**,imageGL.width(), imageGL.height(), 0, **GL_RGB**, GL_UNSIGNED_BYTE,0 );
glBindTexture(GL_TEXTURE_2D, 0);

oh yes is juste the adresse and I try to file it with the datas but memcpy() here is correct so ?

ok so theye are many possibility about INVALID_OPERATION but the problem seems from the buffer object:
or he is mapped
or the memory reads exceed
or the data​ is not divisible into the number of bytes

are there any debug fonction to check the différent state of my buffer object ?

I’m sorry I’m new in OpenGL but I become to get the logic.

what mean the size of my qtImage.bits equal 4 ? like 4 bits by color ?

[Dark_Photon]

@sacha

    glGenTextures(1,&texture);
    glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP);
    glTexImage2D(GL_TEXTURE_2D, 0, **GL_SRGB8**,imageGL.width(), imageGL.height(), 0, **GL_RGB**, GL_UNSIGNED_BYTE,0 );
    glBindTexture(GL_TEXTURE_2D, 0);

You’re missing a glBindTexture( GL_TEXTURE_2D, texture ) right after your glGenTextures() call. Generating texture handle(s) doesn’t bind any of them to the context.

[sacha]

Yes but it doesn’t change anything…

What should I check to use a PBO ?
are they mark as mapped or filled with data ?

[Dark_Photon]

Did you fix the problem I flagged above where you were were not filling the mapped PBO pointer but rather filling a heap memory block? (See pPixelsPBO refs in your code. In particular the new unsigned char line, which you should probably delete.)

[sacha]

No I didn’t see any changement

I delated “pPixelsPBO = new unsigned char[imageGL.width() * imageGL.height() * 3];”

Here is the code to file the pixel buffer into the loadImage fonction call each time in the glutDisplayFunc():

 for (int iBuffer = 0; iBuffer < N_MAX_BUFFERS; ++iBuffer)
  {
    // bind pixel-buffer object
    glBindBuffer(GL_PIXEL_PACK_BUFFER_EXT, pboIds[iBuffer]);

    // create buffer's data container
    glBufferData(GL_PIXEL_PACK_BUFFER_EXT, DATA_SIZE(=imageGL.width() * imageGL.height() * 3 ) , NULL, GL_STREAM_DRAW);

    void * pPixelsPBO = glMapBuffer(GL_PIXEL_PACK_BUFFER_EXT, GL_READ_WRITE);//GL_WRITE_ONLY

    if(pPixelsPBO)
    {
      ////pPixelsPBO = new unsigned char[imageGL.width() * imageGL.height() * 3];
      memcpy(pPixelsPBO, imageGL.bits(), DATA_SIZE);//memcpy
    }
    else
    {
      qDebug() << "pPixelsPBO NULL";
    }
    if (!glUnmapBuffer(GL_PIXEL_PACK_BUFFER_EXT))
    {
      qDebug() << "Couldn't unmap pixel buffer. Exiting";
    }
    // unbind pixel-buffer object
    glBindBuffer(GL_PIXEL_PACK_BUFFER_EXT, 0);
  }