OpenGL ICD behavior in Win7

Hello, dear community.

I’m a rookie programmer and the problem i ran into seems too deep for me, so i need help.

I have an issue with OpenGL API interception in Win7 x64 with newest Catalyst (10.11) drivers. My program integrates into OpenGL based CAD application and intercept calls to opengl32.dll for a purpose to inject own graphics in render context during frame rendering. The way it works is quite simple: the CAD application loads my dll and when it calls my startup function i patch an Import Address Table (IAT) of the particular module in the process to intercept some OpenGL function calls from this module and render my own graphics. This is possibly the one and only method to integrate into this specific application, because i cannot rely on other non render-specific function calls.

This simple trick worked fine until i ran into a problem with newest Catalyst drivers in Win7. It seems that in terms of optimization, when CAD application start up, someone (Windows 7 loader, or ATI driver) is patching it, so after loading into memory it is no more calling opengl32 API thru IAT. In those places of code where it should do that, now ‘call qword ptr’ commands have new arguments (i compared asm codes of both loaded and unloaded images of same module). This argument leads to an element of new table, placed somewhere between other loaded modules. This is address table of internal functions inside ‘atio6axx’ module. So now, each opengl call leads to ATI driver directly, skipping opengl32 interleaf. This is truly an optimization issue because every call to opengl32 in my own ‘unpatched’ module after few instructions jumps to corresponded atio6axx function anyway. It’s even more obvious when taking into account a fact that when i run CAD application from debugger, the loaded modules are unpatched, in other words: every call to OpenGL API goes thru IAT, as it supposed to do and no optimization performed. I also noticed that there are some jumps to opengl32 functions inside CAD application. These jumps are ‘jmp qword ptr’ commands with argument being an element of IAT too. And these jumps are not patched when loaded to memory. So, it seems, ‘optimizator’ only patches ‘call’ commands.

The thing that surprises me most, is when i attach to the process with debugger, that specific ati address table resets to the addresses of original opengl32 functions. This costed me few weeks of understanding why hooking directly opengl32 module with hooking libraries wont work if debugger hasn’t attached yet, until i started to inspect process memory dumps. I also noticed that inside atio6axx functions there are instructions, using variables from Cg module, also loaded in memory, but rarely used. What is it? Cg hooking ATI driver, or vice versa?

So my questions are:

  1. Is it vendor specific optimization, or should i look inside Windows 7 DWM and WDDM specs to deal with it?

  2. Is there a way to turn that optimization off? Switching Catalyst A.I. options didn’t help.

  3. Is there a way to locate this specific address table of ATI functions and find out which function represents which opengl32 call?

  4. Is there another methods to intercept OpenGL calls, or to find out it was supposed to be called?

I would appreciate any your suggestions or help. Thank you in advance and forgive me my bad English :slight_smile:

Try asking on AMD’s development forums. There is a slightly higher chance you might find someone who can help you there.

Out of curiosity, does gDEBugger work with that application and those specific drivers?

I’ve already posted on AMD forums, now waiting for any reply.

And yes, gDEBugger works fine with this CAD application and drivers because it can’t attach to a loaded process, it only works if loading application himself, putting specific opengl32 module in process memory in loading time. And no ‘patching’ occur in this case - this opengl32 module have different TimeDateStamp than module in ‘WinSxS’ directory and ‘patcher’ stay silent.

I have had reports of similar weird patching taking place with Quadro cards (while attempting to use my GLIntercept tool)

I’ve been told on AMD dev forum that this is indeed a vendor specific optimization, implemented by several vendors already though. Sadly, the only solution they provided is to load own opengl32 module before application create first OpenGL context. In that case ICD will detect the change and skip patching. No driver specific commands, options or other solutions provided yet.

Right before they answered me, i’ve managed to trace the moment when this patching occur. It seems, that right after application calls first OpenGL instruction (in my case it is wglChoosePixelFormat) several ‘ati****.dll’ modules loads in memory and patching occur. So, the only reasonable solution i see is too hook one of theese ‘starter’ commands and when they called simply unload opengl32 module and load new one, from your own directory via specific kernel commands (that allows to skip WinSxS directory when search).

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.