virtual object occluded by real object

hi guys, does anyone interest in Augmented Reality? I’m trying to occlude my virtual object created by openGL with the real one. what I’m trying to achieve is like this one Real object occlusion on augmented reality objects - YouTube

I’m pretty sure that by using the depth buffer I can occlude the virtual with the real one. But after conducting numerous trials, I havent succeed. Does anyone want to help me to figure out?

thanks

How you plan to acquire the depth buffer of the real scene? This is a non-trivial problem that most of the times requires multiple cameras to reproject the images captured by the cameras into 3D space.

In the video you’ve linked probably doesn’t perform a true reprojection, but simply handles this special scene, i.e. uses the knowledge that the occluder object is circle shaped and uses a hardcoded value to define it’s distance (or recalculate the distance based on knowing the size of the object). That could be easily done by doing a Hough transform to determine the position and size of the circle representing the occluder, however, in general case such an approach won’t work.

Provided you know the position/shape of the real occluder, it is only a matter of render video background, render virtual occluder with a depth buffer and color mask so that color buffer is not updated, then virtual objects with depth testing normally.

thanks for replying.

zbuffer, can you write the pseudocode for obvious explanation?

thanks

I am curious, how do you get the positions of the real world objects?

[QUOTE=deduu10;1238626]thanks for replying.

zbuffer, can you write the pseudocode for obvious explanation?

thanks[/QUOTE]

glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
renderOccluders();
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
renderOccludees();

This is what zbuffer explained and that’s the only OpenGL related code needed to implement what he proposed. You have to figure out the rest.

[QUOTE=aqnuep;1238645]

glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
renderOccluders();
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
renderOccludees();

This is what zbuffer explained and that’s the only OpenGL related code needed to implement what he proposed. You have to figure out the rest.[/QUOTE]

kopelrativ, I use marker to get the position of the real occluder.

aqnuep, thanks for writing the code. I have ever tried that code before, my virtual occluder succeed occlude my virtual object, but when I used colormask, indeed my virtual occluder was not rendered, but it was occluded by my virtual object. I will keep trying another way. Thanks

Well, then you’ve probably done something wrong. Let me rephrase then the explanation:

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

glDisable(GL_DEPTH_TEST);
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);

render real world picture as a full screen quad

glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);

render simple geometry (e.g. sphere) that acts as the virtual occluder

glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);

render your virtual objects (whatever it be) that acts as the virtual occludee

If you follow this method, it has to work, otherwise there may be a different problem with your code.

[QUOTE=aqnuep;1238660]Well, then you’ve probably done something wrong. Let me rephrase then the explanation:

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

glDisable(GL_DEPTH_TEST);
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);

render real world picture as a full screen quad

glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);

render simple geometry (e.g. sphere) that acts as the virtual occluder

glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);

render your virtual objects (whatever it be) that acts as the virtual occludee

If you follow this method, it has to work, otherwise there may be a different problem with your code.[/QUOTE]

thanks for your explanation, I had tried this one also… the problem is my real scene was cleared to black as I put GL_COLOR_BUFFER_BIT to glClear(). These code is supposed to work if I know how to maintain my real scene…

Presenting your real scene goes to the line I’ve noted as “render real world picture as a full screen quad”, not before glClear. That way it cannot clear your image.

I have no idea how to do that. For building AR I use ARToolkit library. the real scene is automatically appeared, when the init routine (in ARToolkit) is called . Inside the init routine there are function to create window, open video path, and etc. thanks

Well, if you already have your image in the OpenGL back buffer then just proceed with the steps after the the “render real world picture as a full screen quad” step then.

Hii Aqnuep,

if you don’t mind, I wanna ask another question. Consider I have two objects, let’s say a cube and a sphere. I wanna occluded the sphere with my cube. But before doing that, I wanna changed every pixel of the cube to be the pixel of my backgroud. therefore, when my cube occluded my sphere, it displayed the color of my background with the shape of cube. Can you help me how to do that?

thanks

Sorry but I don’t understand completely what is your scenario.

So you have a real world image, having a cube in it and a sphere rendered? You want the real world cube to hide the sphere or the other way around? Also, what do you mean by the “color of your background”? You mean the real world image pixels?

[QUOTE=aqnuep;1239265]Sorry but I don’t understand completely what is your scenario.

So you have a real world image, having a cube in it and a sphere rendered? You want the real world cube to hide the sphere or the other way around? Also, what do you mean by the “color of your background”? You mean the real world image pixels?[/QUOTE]

Yes, I did. I said the color of my background e.g black, and did not talk about the real scene to simplify the scenario. it is exactly of what you said. I have a real world image, consists of real cube, virtual cube and sphere. as the virtual cube has been accurately registered to real cube, the projective position of virtual cube in the image plane is known. therefore, before occluding the sphere with the cube, I want to change every pixel of the virtual cube with the real one. after that, by making use of OpenGL depth buffer, every pixel of virtual sphere which depth value is less than 1.0 will be replaced by the pixel from the real cube’s image.

You don’t need to redraw the pixels of the cube, simply don’t render it to the color buffer in the first place. When you render your virtual cube, simply disable all components of the color masks as I specified in an earlier post. That’s all. Then when you render your cube, only the depth buffer is updated but you don’t update any pixels of the screen. Then re-enable the color mask when rendering the sphere so that those pixels will get their way to the color buffer too.

Well, understood… but, if I render the sphere after the cube, the depth value of my sphere is less than the cube so that the real scene pixel (from disabling glColorMask) will be occluded by the pixel of the sphere. Is it true according to my knowledge? Please correct me if i’m wrong… thanks.