Best practice for mipmapping normal maps?

When generating normal map mipmaps, I use a Kaiser filter and renormalize the values at each pixel of each mip level. I’m also using BC5 compression. While the results are better than standard mipmapping with DXT compression, I’ve noticed artifacts on the lower mipmap levels. The surface roughness either gets washed out or dark spots appear on the lower mip levels. The switch between mip levels is distracting. This is without specular turned on.

Is there a “best practice” for generating normal map mipmaps?