Transform Feedback with double buffer - Garbage data?

Hi all!

I have implemented some nice particle system using Transform Feedback using double buffer, based on the tutorial 28 from OglDev (Tutorial 28 - Particle System using Transform Feedback).

The problem is that after observing some artifacts in the render, I’ve seen that in the process of copying values from one buffer to another, some data with garbage is still there from the previous frame.

Let me explain the issue with an example. Let’s imagine we have 2 buffers that contains room for 5 particles each. Each particle is composed by:

int Type;      // 1=particle emitter, 2=normal particle
float ID;      // a unique ID (in float format). In case it's an emitter we just copy the global time here, so if it's an emitter the ID should be always different every frame (I made this for debug purposes only). In case is a normal particle, the ID is a random number generated on the particle creation, and kept forever
vec3 Position; // particle position
float Life;    // Particle's life (increased every frame)

In my scene I should Always have ONLY 1 emitter, but in the logs I can see this:

My theory is that Buffers are never cleared, so, if during the transition of Frame 2 to Frame 3, we just need to draw 4 particles (3 particles+1 emitter), then the 5th particle of the buffer will be simply not deleted, and will be present, therefore, the particle with ID 3.454 will be still present (which is an emitter), so, in Frame 3 I will have 2 emitters: the correct one (ID=3.553) and an invalid one (ID=3.454) which is basically garbage from the Frame 1.

I am correct?? Is there any way to prevent this?

How are you deleting particles? Transform feedback captures the vertices of each primitive generated, before culling and clipping. If you aren’t using a geometry shader or tessellation shader, then the number of vertices written to the transform feedback buffer(s) is exactly the number of primitives generated by the draw call multiplied by the number of vertices per primitive. A vertex shader cannot change the number of vertices captured.

If particles have a fixed lifespan, then the client code should know which particles are valid without looking at the particle data.

1 Like

Thanks GClements!

I am using a geometry shader for the particle generation, in fact this is the complete shader I’m using:

Vertex shader:

#type vertex
#version 330

layout (location = 0) in vec3 Position;
layout (location = 1) in vec3 Velocity;
layout (location = 2) in vec3 Color;
layout (location = 3) in float Age;
layout (location = 4) in int Type;
layout (location = 5) in float ID;

out VS_OUT
{
	vec3	Position;
	vec3	Velocity;
	vec3	Color;
	float	Age;
	flat int		Type;
	float	ID;
} vs_out;

void main()
{
	vs_out.Position = Position;
	vs_out.Velocity = Velocity;
	vs_out.Color = Color;
	vs_out.Age = Age;
	vs_out.Type = Type;
	vs_out.ID = ID;
}

Geometry shader:

#version 330

layout(points) in;
layout(points) out;
layout(max_vertices = 30) out;

// Info from the VS
in VS_OUT
{
	vec3	Position;
	vec3	Velocity;
	vec3	Color;
	float	Age;
	flat int		Type;
	float	ID;
} gs_in[];

// Info sent to FS
out vec3 Position1;
out vec3 Velocity1;
out vec3 Color1;
out float Age1;
flat out int Type1;
out float ID1;

uniform float gDeltaTime; // Time between frames
uniform float gTime; // Global time, from 0 to inf
uniform vec3 gForce; // global Force, for example: wind or gravity
uniform vec3 gColor; // global Color
uniform sampler1D gRandomTexture; // A texture with white noise
uniform float fEmissionTime; // Time between emission, for example: if it's 0.5, means that every 1/2 second a particle will be generated from the emitter
uniform float fParticleLifetime; // Lifetime of the particle

#define PARTICLE_TYPE_EMITTER 1
#define PARTICLE_TYPE_SHELL 2

uniform mat4 model;

// Get a random number
vec3 GetRandomDir(float TexCoord)
{
	vec3 Dir = texture(gRandomTexture, TexCoord).xyz;
	Dir -= vec3(0.5, 0.5, 0.5);
	return Dir;
}

void main()
{
    float Age = gs_in[0].Age + gDeltaTime; // Increment the age of the particle
	vec3 DeltaP = gDeltaTime * gs_in[0].Velocity; // Position Delta: xDelta = v*t
		
	if (gs_in[0].Type == PARTICLE_TYPE_EMITTER) {
		// If it's time to create a new particle shell...
		if (Age >= fEmissionTime) {
			Type1 = PARTICLE_TYPE_SHELL;
			
			// Apply model matrix to the generated particle
			vec4 new_pos = model*vec4(gs_in[0].Position + DeltaP, 1.0);
			Position1 = vec3(new_pos.x/new_pos.w, new_pos.y/new_pos.w, new_pos.z/new_pos.w );
			
			float randomNum = (gTime + gs_in[0].Position.x + gs_in[0].Position.y + gs_in[0].Position.z);
			ID1 = randomNum; // Assign a random ID to the generated particle
			Velocity1 = gs_in[0].Velocity;
			Color1 = gColor;				//Apply the global color
			Age1 = 0.0;
			EmitVertex();
			EndPrimitive();	// Generate a new particle from the launcher position
			Age = 0.0;		// Set the age of the emitter to 0, so it can generate new particles later
		}	
		// Draw the Emitter
		Type1 = PARTICLE_TYPE_EMITTER;
		Position1 = gs_in[0].Position;
		Velocity1 = gs_in[0].Velocity;
		Color1 = vec3(1,0,0);//gs_in[0].Color;
		Age1 = Age;
		ID1 = gTime;//Assign the global time to the Emitter ID (this is for DEBUG PURPOSES ONLY, ideally the ID should not be changed)
		EmitVertex();
		EndPrimitive();		// Generate the emitter
    }
	
	// If its a normal particle...
	if (gs_in[0].Type == PARTICLE_TYPE_SHELL) {
		
		// If the is still alive, we update the values...
		if (Age < fParticleLifetime) {
			vec3 DeltaV = gForce * gDeltaTime; // vDelta = accel*tDetla

			Type1 = PARTICLE_TYPE_SHELL;
			Position1 = gs_in[0].Position + DeltaP; // x = x0 + xDelta
			Velocity1 = gs_in[0].Velocity + DeltaV; // v = v0 + vDelta
			Color1 = gs_in[0].Color - vec3(1.0, 1.0, 1.0)*gDeltaTime/fParticleLifetime;

			Age1 = Age;
			ID1 = gs_in[0].ID;

			EmitVertex();
			EndPrimitive(); // Update the particle status and position
		}
	}
}
#version 330
void main()
{
}

There’s another shader, which do the billboarding process, but it’s not relevant, because that shader just draw’s what we have on the buffer-

Buffers aren’t cleared unless you explicitly clear them. If the number of primitives generated is fewer than the buffer has space for, the end of the buffer will remain untouched.

You need to count the number of primitives generated, using a query object for GL_TRANSFORM_FEEDBACK_PRIMITIVES_WRITTEN. The count parameter for any draw calls using the generated data should be based upon the number of primitives generated, not the number the buffer has space for.

1 Like

That’s the key. Buffers are remaining untouched… In order to clear buffers I need to Map the buffer and clear it manually? or there is a fastest way to do it?

So maybe I could read the primitives written, and clear the buffer only from the number of primitives written to the Buffer length… so, if I wrote only 4 primitives, and my buffer has storage for 5, I should clear from 4 to 5… is this the best way to do it?

The best way to do it is to keep track of how many elements are in the buffer and ignore the garbage data. If the update pass reports N primitives written, then the draw call to render the data and the draw call for the next update pass should only render N primitives.

If you actually need to clear a portion of the buffer, use glClearBufferSubData. Mapping the buffer risks synchronisation and a pipeline stall.

xphere, also keep in mind that there are more efficient ways to communicate this to the resulting draw call than reading this count back to the CPU (slow!) and populating it manually on the CPU-side.

Take a look at ARB_query_buffer_object and ARB_indirect_parameters

It isn’t strictly necessary to use extensions to avoid the round trip. 4.5 has glGetQueryBufferObject which stores the query result in a buffer. That can be part of a DrawArraysIndirectCommand structure for glMultiDrawArraysIndirect (with a drawcount of 1).

A more direct alternative is to use transform feedback objects (4.0+ or ARB_transform_feedback2) and glDrawTransformFeedback. The latter is equivalent to glDrawArrays with the count taken from the transform feedback object (aside: there appears to be no way to query this count, and it isn’t listed in the state table for transform feedback objects, but it’s stated that it’s stored there).

But if you’re using 3.3, you’ll either need to use extensions or take the possible synchronisation hit. You should be able to avoid explicit synchronisation by checking whether the result is available using glGetQueryObject with QUERY_RESULT_AVAILABLE. If the draw call to render the particles immediately follows the transform feedback pass, you’re effectively still synchronising as the pipeline will empty before the result is available and you can’t start to refill it (with the draw command) until you have the result. But that’s true even without a round-trip through client memory. Assuming that the particles aren’t the entire scene, you should try to fit other operations between updating the particles and rendering them.

I’ll check this, sound promising! :slight_smile:

I’m using query objects so I’t shouldn’t be a big deal to get the number of generated particles. (see code below)

Yes, you can query the Primitives drawn by the transform Feedback with the query: GL_TRANSFORM_FEEDBACK_PRIMITIVES_WRITTEN

Well… I’m using openGL 4.5… but for some reason the shaders are on v3.3… my mistake! I will correct it asap!

FYI, the update/render methods that I use are:

	void ParticleSystem::Render(float deltaTime, const glm::mat4& model, const glm::mat4& view, const glm::mat4& projection)
	{
		m_time += deltaTime;

		glBindVertexArray(m_VAO);
		UpdateParticles(deltaTime, model); // Calls the Update shader and update particle's buffers
		RenderParticles(model, view, projection); // Render the particles of the buffer, using the billboard shader
		glBindVertexArray(0);

		m_currVB = m_currTFB;
		m_currTFB = (m_currTFB + 1) & 0x1;
	}

    void ParticleSystem::UpdateParticles(float deltaTime, const glm::mat4& model)
	{
		m_particleSystemShader->use();
		m_particleSystemShader->setValue("model", model);
		m_particleSystemShader->setValue("gTime", m_time);
		m_particleSystemShader->setValue("gDeltaTime", deltaTime);
		m_particleSystemShader->setValue("gRandomTexture", RANDOM_TEXTURE_UNIT); // TODO: fix... where to store the random texture unit?
		m_particleSystemShader->setValue("fEmissionTime", m_emissionTime);
		m_particleSystemShader->setValue("fParticleLifetime", m_particleLifeTime);
		m_particleSystemShader->setValue("gForce", force);
		m_particleSystemShader->setValue("gColor", color);


		bindRandomTexture(RANDOM_TEXTURE_UNIT);

		glEnable(GL_RASTERIZER_DISCARD);	// Stop drawing on the screen
		glBindBuffer(GL_ARRAY_BUFFER, m_particleBuffer[m_currVB]);

		glBindTransformFeedback(GL_TRANSFORM_FEEDBACK, m_transformFeedback[m_currTFB]);

		// Use Binding for particles (all attributes)
		glBindVertexBuffer(BINDING_UPDATE, m_particleBuffer[m_currVB], 0, sizeof(Particle));

		glBeginQuery(GL_TRANSFORM_FEEDBACK_PRIMITIVES_WRITTEN, m_queryPrimitives);

		glBeginTransformFeedback(GL_POINTS);

		if (m_isFirst) {
			glDrawArrays(GL_POINTS, 0, m_numEmitters);
			m_isFirst = false;
		}
		else {
			glDrawTransformFeedback(GL_POINTS, m_transformFeedback[m_currVB]);
		}

		glEndTransformFeedback();

		glEndQuery(GL_TRANSFORM_FEEDBACK_PRIMITIVES_WRITTEN);
		glGetQueryObjectuiv(m_queryPrimitives, GL_QUERY_RESULT, &m_numParticles);

		glBindTransformFeedback(GL_TRANSFORM_FEEDBACK, 0);
	}

	void ParticleSystem::RenderParticles(const glm::mat4& model, const glm::mat4& view, const glm::mat4& projection)
	{
		//Use the billboard shader and send variables
		m_billboardShader->use();
		m_billboardShader->setValue("model", model);						// Set Model Matrix
		m_billboardShader->setValue("view", view);							// Set View Matrix
		m_billboardShader->setValue("projection", projection);				// Set Projection Matrix
		m_varsBillboard->setValues();

		glDisable(GL_RASTERIZER_DISCARD);	// Start drawing on the screen
		glBindBuffer(GL_ARRAY_BUFFER, m_particleBuffer[m_currTFB]);

		// Use binding for billboard (only Position and color attributes)
		glBindVertexBuffer(BINDING_BILLBOARD, m_particleBuffer[m_currTFB], 0, sizeof(Particle));

		glDrawTransformFeedback(GL_POINTS, m_transformFeedback[m_currTFB]);
	}

Right. The reason I point out functionality with an extension link is that the extension is a self-contained description of a specific added feature, often (as in this case) with sample GL code included.

I should probably have also (or instead) linked to the description of this in the OpenGL Wiki:

Sure, you can use an explicit query. But this value is already stored as part of the transform feedback object, and is used by glDrawTransformFeebacks, but can’t be read via glGetTransformFeedback (or any other means).

After all your suggestions, the solution to the problem was much simpler… a bug on my code :frowning:

Explanation: I have 2 bindings (one for the update shader, and one for the billboard shader), and the billboard binding was incorrect… therefore, the garbage was introduced in the buffer…

But thanks everyone for the tremendous help! all your suggestions give me lots of nice ideas and improvements for the future!!

Big thumbs up for this great community!!!