The only way i know of embedding OpenGL inside IE is by using an ActiveX control. I don’t know how to program those really well, but if you want to find out, check out ATL. You can do it with MFC but the result is a big control that isn’t really suitable to be on the web (large loading times). If you use ATL, its a bunch of templates, so the result will be compact and efficient. If you keep scrolling down, you will find an example of OpenGL in an ActiveX control. It’s really cool!!! I can definetely see why you want to do this. Though i warn you, ActiveX controls aren’t exactly easy from what i’ve seen. Here is the link to the site with the example (scroll down to find it):
Or you could check out gl4java at http://www.jausfot.com/gl4java
which allows applets to use java based opengl libraries.
make that http://www.jausoft.com