Rendering artifacts when normalizing normal map

[aferreira]

Hello!

I am having trouble when trying to sample a normal map for terrain. So when i do this:

void calculateNormal(vec2 texCoord, mat4 mMatrix)
{	
	vec3 normal = texture(TexTerrainNormal, texCoord).xyz;
	normal_GS_in = normal;
}

i get this, which looks correct:

But when doing:

void calculateNormal(vec2 texCoord, mat4 mMatrix)
{	
	vec3 normal = texture(TexTerrainNormal, texCoord).xyz;
	normal_GS_in = normalize(normal);
}

or

void calculateNormal(vec2 texCoord, mat4 mMatrix)
{	
	vec3 normal = texture(TexTerrainNormal, texCoord).xyz * vec3(2.0) - vec3(1.0);
	normal_GS_in = normalize(normal);
}

i get this glitchy mess:

Also, the black spots in the terrain flicker.

I am sampling the normal texture in the TES and passing it through all the way to the pixel shader, through the geometry shader in all situations.

Any ideas what could be wrong?

Cheers,
André

[aferreira]

Also, i am getting very weird results when manipulating that first image. For example i have a tonemap shader:

void main()
{
    // Retrieve color from texture
    vec4 hdrColour = texture(lColorTexture, ftexcoord);
    vec3 bloomColor = texture(bloomBlur, ftexcoord).rgb;

    hdrColour.rgb += bloomColor; // additive blending
    
    // Apply toneMapping
	// HDR tonemapping
    vec3 LDR_Color = hdrColour.rgb / (hdrColour.rgb + vec3(1.0));
	
    // gamma correct
    LDR_Color = pow(LDR_Color, vec3(1.0/2.2)); 
	
    fragColor = vec4(LDR_Color, hdrColour.a);//hdrColour;//ToneMap(original_fragColor);
}

With this i am getting NaN values to the output texture, when having a perfectly valid input color texture and the bloomBlur being 0.

Weird thing is that none of this happens on another AMD card (R9 280x) and happens on a much newer 5700XT

[aferreira]

Well this keeps getting weirder. If i dont comment this line in the TES shader:
tes_tbnMatrix = tbnMatrix;
it breaks, if i comment it “works”
Problem is that i dont use this variable anywhere in the code, it just pass through until the pixel shader where i literally have just this in the main()

void main(void)
{
	FragColour.rgb = gs_normal;
	return;
}

[aferreira]

Solved by moving all normal calculations to the geometry shader. Dont know why though.