I’m currently working on a translation of a HLSL shader to a GLSL shader. The HLSL shader is a shader for Total War: Attila, provided in the official modding tools and useable in the 3ds Max viewport. I have started working on a translation from the HLSL file to a GLSL file which can be loaded as a custom shader in Substance Designer 5.6.2 to create a better synchronized workflow during the texture creation process.
In the beginning I had some application-specific problems which I managed to solve with the help of the Allegorithmic product support: https://forum.allegorithmic.com/index.php/topic,16074.msg67636.html
Now I’m at a point where all base functions are implemented and useable in the GLSL shader but I’m really new to shader programming (however, I think I learn fast because of my advanced knowledge in programming itself). The initial HLSL file contains about 2500 lines of code. So that doesn’t make things easier unfortunately. All in all, I went through the whole Direct3D code and translated it to the equivalent OpenGL statements. I had to fix some matrix calculations and some minor other things but the the GLSL shader is already capable to read all possible kinds of game textures. In addition the TBN matrix (which actually is a mat3(tangent, normal, bitangent) matrix) should work fine as well as the normal map is represented correctly in Substance Designer’s 3D view.
However, the core problem now is that I get completely different results with the original HLSL shader in the 3ds Max viewport and the GLSL shader in the Substance Designer 3D view. The light calculations are exactly the same in both shaders (at least as far as I can tell, it might be possible that I messed something up during the translation).
Here’re some pictures to visualize my problem (please have a look into my following posts, I never posted in this forum before so it prevents me from linking multiple pictures).
It looks like my GLSL shader doesn’t really calculate any specular color (Total War: Attila uses RGB specular maps) and for some reasons the light calculation looks broken.
Maybe someone can give me more specific hints what the problem might be? I can attach parts of the source code or the whole one here as well but I don’t know if it’s a good idea since it has more than 800 lines of code right now. Anyway, any help is appreciated and if you need more informatin I will provide it as soon as possible!