Got a weird issue that I’ve been able to drill down into a specific point, but am now hitting a road block. I’ve been working on an engine for awhile and recently noticed that I had an intermittent (drastic) decrease in framerate for a single frame. The first time it happens is almost exactly at 30 seconds into the application runtime every time. I was able to profile the application and discovered that the issue is somehow related to binding textures (or so it seems on the surface), as I could watch the profiler and as soon as the drop happened it was reported that my method responsible for binding textures and passing uniform data was taking up to 80ms to complete (tested multiple times and results varied anywhere between 20ms and 80ms).
I find it strange that the issue is this intermittent. The first time it happens takes 30 seconds, then after that it might do it again, and it might not, but if it does it’s several minutes past the first time it occurs. Other than the first time it happens, there’s no recognizable pattern. Is there anything that anyone can think of that might be causing this behavior? It almost feels like maybe there’s a race condition on the gpu?
Now…what I find very strange…I began debugging with nvidia nsight thinking that whatever was happening would be pretty apparent once it happened, however…the issue is non-existent when the opengl debug context is enabled. I confirmed this by adding application code to enable and print opengl debug context messages directly (removing nsight from the equation) and the issue is non-existent as well. As soon as I disable the opengl debug context, issue presents itself again.
I know there’s probably no quick easy answer to why this is happening, but I wanted to see if anyone else has encountered behavior such as this, or would know of any other means in which I might track down what is going on.
Below is a stripped down version of what’s going on (if anyone would like to see the full codebase it can be accessed via the “massive_refactor” branch in the repo:
// Scene::draw occurs every tick, there's a lot more code within the
//actual method, this is just the relevant part
void Scene::draw(float alpha)
{
// This reports that useHdr is the bottleneck
if (r.getShader() != gpu->getActiveShader())
gpu->useShader(r.getShader());
if (s.getActiveHdr() != gpu->getActiveHdr())
gpu->useHdr(s.getActiveHdr());
if (r.getMesh() != gpu->getActiveMesh())
gpu->useMesh(r.getMesh());
if (r.getMaterial() != gpu->getActiveMaterial())
gpu->useMaterial(r.getMaterial());
// This reports that useMaterial is the bottleneck, note that
// after useShader, we first call useMesh which does not exhibit
// the same behavior
/*
if (r.getShader() != gpu->getActiveShader())
gpu->useShader(r.getShader());
if (r.getMesh() != gpu->getActiveMesh())
gpu->useMesh(r.getMesh());
if (r.getMaterial() != gpu->getActiveMaterial())
gpu->useMaterial(r.getMaterial());
if (s.getActiveHdr() != gpu->getActiveHdr())
gpu->useHdr(s.getActiveHdr());
*/
for (auto& a : r.actors.getAll())
this->drawActor(a, alpha);
}
void GPU::useHdr(HDR* h)
{
this->activeHdr = h;
// bind pre-computed IBL data
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_CUBE_MAP, h->irradianceMap);
this->setShaderInt("irradianceMap", 0);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_CUBE_MAP, h->prefilterMap);
this->setShaderInt("prefilterMap", 1);
glActiveTexture(GL_TEXTURE2);
glBindTexture(GL_TEXTURE_2D, h->brdfLUTTexture);
this->setShaderInt("brdfLUT", 2);
}
void GPU::useMaterial(Material* m)
{
this->activeMaterial = m;
glActiveTexture(GL_TEXTURE3);
glBindTexture(GL_TEXTURE_2D, m->albedo->id);
this->setShaderInt("albedoMap", 3);
glActiveTexture(GL_TEXTURE4);
glBindTexture(GL_TEXTURE_2D, m->normal->id);
this->setShaderInt("normalMap", 4);
glActiveTexture(GL_TEXTURE5);
glBindTexture(GL_TEXTURE_2D, m->metallic->id);
this->setShaderInt("metallicMap", 5);
glActiveTexture(GL_TEXTURE6);
glBindTexture(GL_TEXTURE_2D, m->roughness->id);
this->setShaderInt("roughnessMap", 6);
glActiveTexture(GL_TEXTURE7);
glBindTexture(GL_TEXTURE_2D, m->ao->id);
this->setShaderInt("aoMap", 7);
}
void GPU::setShaderInt(const std::string& name, int value) const
{
// it was mentioned that I may try caching uniform locations in order
// to resolve the issue, while this is a good idea, and something to be
// implemented moving forward, I don't think that would be the root cause
// of this issue (since enabling the debug context resolves it)
glUniform1i(glGetUniformLocation(
this->activeShader->id,
name.c_str()),
value);
}