My application load textures, with transparent areas (alpha=0).
I want to know what is the color under my mouse cursor(RGBA) and if the pointer is not over the texture, return -1.
I see several problems:
1.) When you want to return either the RGBA color or -1 you are aware that -1 is represented as a unsigned int of 0xFFFFFFFF which can also be a real texel color?
2.) You did not mention if you use lighting and shading because that may alter the original textel color.
3.) You can read the final screen color using glReadPixels(), finding the corresponding triangle/quad or whatever you use to diplay the texture would be done by picking.
4.) Perhaps you can explain in more detail how you draw the textures?
So I render a secene with just one texture and with black clearcolor.
My texture has transparent area(alpha=0).
I want to know what is a texture RGBA under my mouse pointer.
I have four globals variable R,G,B,A and I edit them.
If I pick the texture, function return 0, else returns -1.
My problem is that on screen, transparent area appears black (due to OGL clear color). If I do an glReadPixels, I will have (0,0,0,1) or OGL ignores clearcolor and glReadPixels will say me (0,0,0,0) ?