Cannot apply effects on video feed

I am trying to play with convolution shaders like those in the orange book, chapter 19. it’s working fine with pictures but I cannot make it to work with a webcam feed. the image binding is as a sampler2D and all the effects from the shaders work fine, but I can only display my webcam as a sampler2DRect. if I use a sampler2D the background is black (empty) and if I use a sampler2DRect there is webcam feed but the shaders have no effect on it whatsoever, even though they work fine on pictures!

I wonder if this is a non-power-of-two-textures issue I’m having. The pictures are displayed in 256x256 but the webcam cannot be displayed in sampler2D unless I also make it 256x256 or 512x512 ? the sampler2DRect used right now for the webcam is 640x480 so maybe that’s why it doesn’t work as sampler2D ?

please help :slight_smile:

I believe that every card that support GLSL should work with NPOT textures.
However by default 2D textures must have mipmaps otherwise there are undefined. This is not the case with rect, so this is a good lead to investigate. And rect use non-normalized texcoords, so instead of range [0;1] you would have to use ranges [0;n][0;M] for an NxM rect texture.

Try using GL_LINEAR for MINification of 2D texture, see if it makes a difference.

A good idea is to check for GL errors everywhere.

I do not see any related errors. Should I try GL_LINEAR with a sampler2D or sampler2DRect ?

Try GL_LINEAR with a sampler2D of course, with *rect there are no mipmaps.
Read this :
Especially the “filtering” part.

I understand the concept. But I think I am confused on what is the difference between reading the pixel color in an image and in a live webcam feed. Why is the sampler2D not able to get the pixel color out of my webcam? I can only do it with sampler2DRect

The article you were linked to explains half of it. Textures have a type. 2D textures are not the same as rectangle textures. 2D textures are of the texture type GL_TEXTURE_2D. Rectangle textures are of the type GL_TEXTURE_RECTANGLE.

The part you didn’t get was on the page on GLSL samplers. See, samplers mirror the texture type. If you’re using a GL_TEXTURE_2D, you must use the sampler type sampler2D. If you’re using a GL_TEXTURE_RECTANGLE, you must use the sampler type sampler2DRect. You can’t mix and match texture types with the same shader. A different texture type requires a different shader with a different sampler type.

If the texture you use for your live webcam feed is a GL_TEXTURE_RECTANGLE, then the shader that samples from that texture must be using sampler2DRect.

Alfonse, thanks for the samplers link. I was using this block for creating a rectangular texture to hold my webcam feed:

glTexImage2d(GL_TEXTURE_RECTANGLE_ARB, 0, internalformat, width, height, 0, format, type, data);

what is the way to create a GL_TEXTURE_2D for the same reason, to hold my webcam feed? just replace GL_TEXTURE_RECTANGLE_ARB in the two lines with GL_TEXTURE_2D ?

does anyone have an example of applying any kind of GLSL shader effect to a webcam feed ?

A webcam feed is not special, as far as OpenGL is concerned. Where the pixel data for the texture comes from, whether it’s a webcam or a file or whatever, is irrelevant. It’s all about “I have a texture, I need to fetch from it in GLSL and do stuff.”

I’m not sure I understand the problem. You seem to have it working with a rectangle texture. Is there some reason you need to use a 2D texture?

the only reason I need to do it is because the orange book has all the examples in 2D textures. I cannot use any of the shader effects with the Rectangle texture. I can still see the webcam feed but it’s not altered at all by the shaders :frowning:

I only need to see a simple example of a shader effect working on a live webcam feed so I can understand how to adapt the ones in the orange book

the only reason I need to do it is because the orange book has all the examples in 2D textures. I cannot use any of the shader effects with the Rectangle texture.

Wait a minute.

You have a shader that is capable of accessing textures built from your live webcam feed via a texture rectangle. And the Orange book examples use 2D texture for “shader effects”. And you can’t merge the two yourself?

This is a textbook example of why copy-and-paste coding never works. You do not need to use a 2D texture; you have perfectly functioning code already. You just need to make your program perform the post-processing effects you need it to do.

And you can’t do that by copying and pasting code. Those Orange book examples are not there to be copied verbatim into your app; they’re there as object lessons for you to learn from.

If you have an application that can display webcam stuff through a texture rectangle, just write the shader code that uses the texel data you’re fetching from that rectangle texture to do the post-processing effect you need.

Alfonse, who said anything about copying anything? I just said I cannot make ANY shader I could imagine to work on Rectangular textures. For some reason I do not understand why the pixel color is read in a way for the 2D texture and in another way for the Rectangular one. That’s all I said in all my posts. There is not even ONE example available on the web for Rectangular texture and shader effects applied on it, so how can I trust it will ever work?

Yes !!! this made all the difference :slight_smile:
I cannot believe such a small detail could change everything. how did you figure it out ? what is the reason this helped ?

Come on.
It is because as already said, the initial MIN needs mipmaps to work (it is GL_NEAREST_MIPMAP_LINEAR), read this :

Not the case with RECT because this texture format is more limited and can not support mipmaps :

now I see I missed that you also said this:

which obviously I didn’t read until now :slight_smile:
thanks a lot Zbuffer !

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.