Render to Texture issue (SDL+OpenGL)

<INTRO attrib=“Skippable”>
Here is a thread on gamedev.net which has recieved approximately 0 helpful responses… It’s possible I have missed some helpful advice though from one of the three posters who has responded (this is over many days and many problems which I solved myself through much trial and error.)

I’m not looking to be spoon fed, but I really feel like I’ve hit a bit of a wall in my sphere of understanding and I can’t find any references describing a similar issue… So I have turned here in hopes of help.
</INTRO>

Alright, so my current issue is this: I am rendering a 2d scene. On this 2d scene I am drawing a texture first, and then another texture on top of that. Let’s call the lower texture the canvas and the upper texture the brush.

-Both textures have an alpha value.

-I clear the screen to RGBA = 0, 0, 0, 0

-I render both textures in proper order (back to front.) When I render the lower texture I disable blending, when I render the brush texture I enable blending.

-I then update the lower texture with the current screen’s updated scene render at the proper location. I do this with glCopyTexImage2d for GL_RGBA.

I figured this should be a good way of rendering a paintbrush drawing over a surface, but I’m having a weird blend issue.

I’ll describe the issue first: Somehow I am rendering to the screen and it looks proper, but when I call the glCopyTexImage2d function with GL_RGBA it is taking alpha values around the brush and is copying those as well. So if my brush has partial transparancy around it then the alpha values seem to add up on eachother until eventually it’s completely clear behind that location of the texture (if I keep rendering the texture in the same place… otherwise it just adds the alpha values until I “move” the brush and might leave it at only a partial transparancy).

Let me say though, that when calling glCopyTexImage2d with just GL_RGB there is no problem with this. the image captured has basically the blending which appears on screen and it works just fine. Except of course, I want to capture the alpha values of the canvas because if I were to draw something under the thing, the black of the screen is captured too which makes that impossible without obscuring the object.

Here is relevant code (Note, these are just disembodied functions of course, meaning this is not my whole project, just a few functions I use):

//SETUP CODE FOR SETTING UP SDL AND OPENGL.
//Width and Height refrence the pixel width and
//height of the window

bool Draw::setupSDL(int width, int height){
   // Flags we pass into SDL_SetVideoMode. We want an OpenGL window,
	// but we'll let SDL pick the best value of bits-per-pixel.
	// Forcing SDL to use a different BPP than your desktop will
	// slow it down.
	 
   int flags=SDL_OPENGL|SDL_ANYFORMAT|SDL_SRCALPHA;

	if( SDL_Init(SDL_INIT_VIDEO) < 0 ){
      // Failed, exit
      fprintf( stderr, "Video initialization failed: %s
", SDL_GetError());
	   return(0);
   }

   if(flags&SDL_FULLSCREEN){
      SDL_GL_SetAttribute( SDL_GL_DOUBLEBUFFER, 1 );
   }
   SDL_GL_SetAttribute( SDL_GL_RED_SIZE, 8 );
   SDL_GL_SetAttribute( SDL_GL_GREEN_SIZE, 8 );
   SDL_GL_SetAttribute( SDL_GL_BLUE_SIZE, 8 );
   SDL_GL_SetAttribute( SDL_GL_ALPHA_SIZE, 8 );

   SDL_GL_SetAttribute( SDL_GL_ACCUM_RED_SIZE, 16 );
   SDL_GL_SetAttribute( SDL_GL_ACCUM_GREEN_SIZE, 16 );
   SDL_GL_SetAttribute( SDL_GL_ACCUM_BLUE_SIZE, 16 );
   SDL_GL_SetAttribute( SDL_GL_ACCUM_ALPHA_SIZE, 16 );

   SDL_GL_SetAttribute( SDL_GL_DEPTH_SIZE, 16 );

    if( SDL_SetVideoMode( width, height, 32, flags) == NULL ){
        fprintf( stderr, "Video mode set failed: %s
", SDL_GetError( ) );
		  atexit(SDL_Quit); // Quit SDL at exit.
        return(0);
    }
    return 1;
}

void Draw::setupOpengl(int width, int height){
   double ratio = (double) width / (double) height;

   glShadeModel( GL_SMOOTH ); // Our shading model--Gouraud (smooth).

   glViewport( 0, 0, width, height );   // Setup our viewport. 

   // Change to the projection matrix and set
   // our viewing volume.
	glMatrixMode(GL_PROJECTION);
	glLoadIdentity();

   glOrtho(0, width, height, 0, -15.0f, 15.0f);
	
	glMatrixMode(GL_MODELVIEW);
	glLoadIdentity();
   //gluPerspective( 60.0, ratio, 1.0, 1024.0 );

	glClearColor(0.0f,0.0f,0.0f,0.0f);
	glClearDepth(15.0f);
   

   glEnable (GL_BLEND);
   glDisable (GL_ALPHA_TEST);

	glDisable(GL_DEPTH_TEST);
	glShadeModel(GL_SMOOTH);
	glDisable(GL_CULL_FACE);
	glHint(GL_PERSPECTIVE_CORRECTION_HINT, GL_NICEST);
   glHint(GL_SMOOTH, GL_NICEST);
   glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
   glDepthMask(0);

   glEnable(GL_TEXTURE_2D);
   glMatrixMode( GL_MODELVIEW );
   glDisable(GL_LIGHTING);
   glLoadIdentity();
}


//Here is our render to texture command...
//I call "startRenderToTexture" then I render
//the stuff I want to save to the texture
//then I call "endRenderToTexture" to commit
//the changes

void Draw::startRenderToTexture(int x, int y, int width, int height, GLuint textureToBind){
   if(width > 0 && height > 0){
      glDrawBuffer(GL_BACK);
      glViewport(x,y,width,height);					// Set Our View-port (Match Texture Size)
      glClearColor(0.0, 0.0, 0.0, 0.0);
      glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
      RTTcurrent++;
      RTTx.push_back(x);
      RTTy.push_back(y);
      RTTheight.push_back(height);
      RTTwidth.push_back(width);
      RTTtextureToBind.push_back(textureToBind);
      removeClipPane(); //fix clipping with textures later
      glMatrixMode(GL_PROJECTION);
      glPushMatrix();
      glLoadIdentity();
      glOrtho(x, width+x, y, height+y, -15.0f, 15.0f);
      glMatrixMode(GL_MODELVIEW);
   }
}

void Draw::endRenderToTexture(){
   if(RTTcurrent!=-1){
      glBindTexture(GL_TEXTURE_2D,RTTtextureToBind[RTTcurrent]);			// Bind To The specified Texture
      //copy the screen to the bound texture
      glCopyTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, RTTx[RTTcurrent], RTTy[RTTcurrent], RTTwidth[RTTcurrent], RTTheight[RTTcurrent], 0);
      //updateScreen();

      glViewport(0, 0, resX, resY);		// reset the view port to the proper size

      glMatrixMode(GL_PROJECTION);
      glPopMatrix();
      glMatrixMode(GL_MODELVIEW);
      removeClipPane();
      RTTcurrent--;
      if(RTTcurrent < 0){
         RTTcurrent = -1;RTTx.clear();RTTy.clear();RTTwidth.clear();RTTheight.clear();RTTtextureToBind.clear();
      }
   }
   //fprintf(stderr, "end");
}

//The following is the primary loop of the
//function listed.  NOTE: UpdateCanvas is
//a function which just calls startRenderToTexture
//and endUpdateCanvas is a function which calls
//the corresponding endRenderToTexture for the
//appropriate coordinates of the location of
//the texture in question.
//renderer.drawQueued is simply rendering a
//rectangle which I put on my internal sorted
//list (yes, I know, a list being called a queue)
//the rectangle has a texture with alpha
//transparancy.  It is our "brush" and is
//rendered above the "canvas" when it is
//rendered.  The canvas render function
//simply draws a textured rectangle of the
//previous information we had rendered to it.

void Layer::updateLayer(int sX, int sY, int sWidth, int sHeight){
   ... OMITTED CODE FOR DETERMINING WHICH
       TEXTURES WE NEED TO UPDATE BASED ON
       THE ABOVE PIXEL COORDINATES ...
   for(;startVertical <= endVertical;startVertical++){
      for(startHorizontal=origStartH;startHorizontal <= endHorizontal;startHorizontal++){
         identifierName = name+"|";
         tmpStr = cast_stream<std::string> (startHorizontal);
         identifierName+=tmpStr+"|";
         tmpStr = cast_stream<std::string> (startVertical);
         identifierName+=tmpStr;
         glDisable(GL_BLEND);
         part[identifierName].setCanvasBounds(x+(startHorizontal*textureChunkSize), y+(startVertical*textureChunkSize), part[identifierName].getWidth(), part[identifierName].getHeight(), depthZ);
         part[identifierName].startUpdateCanvas();  
         part[identifierName].setMainSize(1);
         part[identifierName].renderCanvas(x+(startHorizontal)*textureChunkSize, y+(startVertical*textureChunkSize), depthZ, 1.0);
         part[identifierName].setMainSize(0);
         glEnable(GL_BLEND);
         renderer->drawQueued(0, 1);
         part[identifierName].endUpdateCanvas(); 
      }
   }
}

Help :frowning: :frowning: :frowning: :frowning:

Seriously, if anyone can help me out with this… Just… That would be amazing.

screenshots depicting my issue can be found near the bottom of the gamedev.net thread I linked to above. I would suggest taking a look at them.

I’ll mention you in the credits under “special mention” if you can solve this, I will give a link to your site or a choice phrase (and you can say ANYTHING) and it will appear right on the splash screen and it will appear in all versions of the software.

I am using blendmode:
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

you can see in the latest image in that thread the effects of this blendmode… Before I was using glBlendFunc(GL_SRC_ALPHA, GL_ONE); with no alpha issues of this kind… Maybe it’s a problem with my blending.

I don’t know.

Ask if you are seriously interested in helping out, but need the project as is to see.

Here is an example image I just took showing lower alpha values over-riding other alpha values…

Now, when I render the final texture with blending OFF it renders fine (no outlines around the brush texture), the alpha values don’t appear like this… When I render the texture with blending on, it appears with all those weird alpha outlines. I don’t get it. Something weird with alpha values is occuring…

Also, the texture I’m using is a small white image with a circle surrounded by alpha transparancy and little grey inconsistancies within the white circle

The top left circle is just another textured quad rendered underneath the whole thing to show that the background alpha is working…

The problem with the black borders around your brush stems from texture filtering. The problem is caused by putting a white circle on black backgroud. Even though the background has alpha = 0, and thus should be invisible. When texture magnification and linear filtering come into play, however, you suddenly start mixing white, opaque pixel with black transparent pixel. This yields some half-transparent /gray/ pixels, instead of half-transparent white pixels, as you would like. To fix this problem, increase the size of the white area in your bitmap by at least one pixel without changing the alpha values, such that your white circle is surrounded by a transparent white border. You can also replace all black parts of the image, of course.

I gave that a try. I achieved it by rendering my brush (at its original resolution rather than the stretched size) to a white, transparant background and then copying it to a texture. First I guess, is that acceptable? I had no perceptable change in result.

Second, what about regular alpha transparancy being rendered over top fully opaque pixels? In the above image I have, the transparancy sort of eats away at the image. What I would rather have is the transparancy stack sort of like stained glass on top of itself. So if you add lots and lots of partially transparant objects it adds up to the original colour of the 100% opaque object.

When calling glCopyTexImage2d with GL_RGB it works, here is an example

(NOTE: I cleaned up my brush texture so that there are no grey splotches in it, so it’s just a white un-anti-aliased circle being expanded by opengl from 32x32 to 100x100)

I may have approached solving the issue you described improperly, but I think I did it right. When calling glCopyTexImage2d with GL_RGB instead of GL_RGBA after doing the render over white transparancy for my brush initially it gives me a square the size of the brush. Essentially I would rather it didn’t matter what color my brush is. If there are partially grey transparant pixels, it should go towards opaque grey pixels as they blend, not towards 100% alpha pixels because it’s partially transparant.

You’re currently using a single blend function for both color and alpha. With glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA), the alpha value does not add up, but is averaged between source and destination just like color. To have alpha add up while keeping the current color blending, you need to provide a separate alpha blend function. See the EXT_blend_func_separate extension.

AFAICS you should be able to get the wanted effect using glBlendFuncSeparateEXT(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE_MINUS_DST_ALPHA, GL_ONE) and initialiting the framebuffer to alpha = 0.

EDIT: If you need your brush texture only to control alpha (ie not changing color), you can also use a grayscale image and upload that into a GL_ALPHA texture. Then apply that texture using GL_REPLACE (to discard glColor’s alpha) or GL_MODULATE (to modulate glColor’s alpha).

You rock. Not just a little, either. Like super hard.

You rock so hard stones can’t even rock as hard.

<3 <3 <3

What can I put you in as? I’d like to mention your name or handle or whatever? memfr0b ok? Something else? And do you want some words of wisdom next to your name or anything? Just let me know. Thank you.

it worked if you couldn’t tell :wink: I had a feeling I just wasn’t blending properly, but I couldn’t figure it out for the life of me… I knew what my problem was, I just didn’t know how to get around it and you provided the perfect solution with a simple extension (and an alternative too)

Edit: link to a website, anything.

Heh, thanks for the flowers :slight_smile: Nice to hear it worked.

If you want to mention my minor contribution to your project, sure, the name “memfr0b” is fine. My website is http://memfrob.de , if you wish to link it. But that’s your call.

A link to your project would be cool though, so i can check it out =)

Oh, definitly. I’ll send you a copy when it’s done.

Those were meant to be hearts, but I’ll give you some flowers too.

-,-<@ -,-<@