I tried posting this as a Github issue, but it was closed and I was told to post here. I can’t link to places or embed more than one thing, though, as I have a new account, so this post is lacking a bit compared to the issue. It’s KhronosGroup/GLSL Issue #165 if you’d like to see a few more pictures of what’s happening. It also has a link to all the code other than the scene vertex/fragment shaders, if that helps.
I’m currently using LibGDX and am trying to render shadows. I have everything working now, mostly based off of a guide from Microbasic called shadow mapping, but when I try and render too many objects at once the shader breaks.
Here are the scene vertex and fragment shaders I’m using:
Vertex:
attribute vec3 a_position;
attribute vec2 a_texCoord0;
attribute vec3 a_normal;
uniform mat4 u_projViewTrans;
uniform mat4 u_worldTrans;
uniform mat3 u_normalMatrix;
varying vec2 v_texCoords0;
varying float v_intensity;
void main()
{
// Vertex position after transformation
gl_Position = (u_projViewTrans * u_worldTrans) * vec4(a_position, 1.0);
v_texCoords0 = a_texCoord0;
// Just add some basic self shadow
vec3 normal = normalize(u_normalMatrix * a_normal);
v_intensity = 1.0;
if(normal.y < 0.5){
if (normal.x > 0.5 || normal.x < -0.5)
v_intensity *= 0.8;
if (normal.z > 0.5 || normal.z < -0.5)
v_intensity *= 0.6;
}
}
Fragment:
uniform sampler2D u_diffuseTexture;
uniform sampler2D u_shadows;
uniform float u_screenWidth;
uniform float u_screenHeight;
uniform float u_antiAliasing;
uniform float u_ambientLight;
varying vec2 v_texCoords0;
varying float v_intensity;
void main()
{
vec4 finalColor = texture2D(u_diffuseTexture, v_texCoords0);
finalColor.rgb = finalColor.rgb * v_intensity;
// Retrieve the shadow color from shadow map
vec2 c = gl_FragCoord.xy;
c /= u_antiAliasing;
c.x /= u_screenWidth;
c.y /= u_screenHeight;
vec4 color = texture2D(u_shadows, c);
// Apply shadow
finalColor.rgb *= (u_ambientLight + 0.6 * color.a);
gl_FragColor = finalColor;
}
When I’m only rendering a few objects, the shader works as expected. However, when I try and render too much, this happens
Additionally, while trying to pinpoint the problem, I discovered that under some circumstances, seemingly random textures from the graphics card can be rendered instead of the shadow. This leads me to believe that the problem may be related to/caused by a problem with random memory access, but I have no idea where that would be coming from.
I was able to cause the weird texture problem by replacing a snippet of code in the fragment shader:
gl_FragColor = finalColor;
with
if (finalColor.r == color.a) {
color.r = color.a;
}
gl_FragColor = color;
Without the if statement, it just shows the shadow map instead of the normal colors. However, adding the if statement (and forcing it to evaluate finalColor, I think otherwise it optimizes it out of the code) causes the problem.
I’ve tried everything I can think of to fix this problem for a few days now, and making a forum post is the last troubleshooting step I can really come up with.