OpenGL Textures and Memory Leaks

r-jaoui · March 24, 2021, 8:05am

Hello !

So I’ll post this question here, I’m not sure if it has to do with OpenGL or with the OpenGL implementation I’m using.

I’m currently writing a 2D Graphics engine, and in a few occasions I seem to get memory leaks when working with textures. Here is my setup :

Images are stored as a OpenGL Texture coupled with a pixel array (the array can be loaded from the texture, and at a later date the texture can be updated with the pixels)
For a few Texture rendering purposes, a single general Texture Rendering FBO is statically created, and is then used with glFramebufferTexture2D each time I want to modify a texture.
My implementation is in Java and Java’s GC doesn’t allow for object destruction. I can get around it with the finalize method, on which I delete the specific texture linked with this Image (and the console shows that textures are deleted). I also tried to create a close function myself, and the leak is still present.
The specific code I’m trying to run is to clear the screen and draw an image (loaded from a file in init), then gather the whole panel as a texture (using glReadPixels to get the pixels and then loading them onto a texture). This seems to be the line that causes the leak, but also just creating an image on each frame (and although it is destructed at the end of the frame) still causes a leak.
When displaying the Texture IDs that are being destructed at each frame, I get something like this :

and at this point the memory used by the programs can be in the Gigabytes…

Is there anything that I could be doing wrong OpenGL-wise (not concerning the JOGL implementation itself, which wouldn’t have its place here) that could cause that ? like launching glGenTextures each time I create an Image (and although they are all eventually deleted).

Thanks !

mhagain · March 24, 2021, 9:34am

This question comes up very regularly. The answer is that glDeleteTextures doesn’t do what you think it does.

More specifically, glDeleteTextures does not free allocated memory; all that it does is make the texture name (i.e. it’s integer handle) available for reuse by a subsequent call to glGenTextures.

The OpenGL specification actually makes very few promises around memory, aside from specifying the occasional GL_OUT_OF_MEMORY error. Otherwise, how memory is handled is very much up to the implementation.

So with that in mind it’s perfectly legal for a GL implementation to free memory immediately, to allocate memory in larger pools and draw down from those pools, to return memory to a previously allocated pool instead of freeing it, to keep memory around to more quickly satisfy a subsequent allocation requirement, some of the above, all of the above, none of the above, or something completely different.

The responsibility is on you to be aware of this and to write your code without assuming that glDeleteTextures will free memory.

r-jaoui · March 24, 2021, 10:08am

Oh… Alright, well at least it isn’t the implementation… I’ll try to look up how JOGL handles Memory issues.

In the meantime there is still another idea I could try, so just one more question : if a specific texture is generated and the a call to glTexImage2D is made, the texture is used and at a later date I want to “override” this texture with another (possibly of a different size) with glTexImage2D, would that work or is it not allowed ? (issue is that although glDeleteTextures are called, the indices still go up which makes me think that at least this implementation doesn’t return the freed indices first). The solution for me would then be to keep a record of the used textures and if possible, reuse these ones instead of creating new unneccessary ids and so virtually the number of actual texture used would simply be the number of concurrent used textures (in my case 2 instead of 400 ^^).

Thanks for your help

Midnightas · March 24, 2021, 10:37am

That could work, but it still may not free old texture memory. I would only do that if for some reason you need your names to stay small (maybe bit-crushing).

r-jaoui · March 24, 2021, 11:38am

Alright, well I posted the same question in the JOGL forums just to know how JOGL’s implementation works (I can’t fine anything online about JOGL’s texture implementation), but in the meantime I’m kind of stuck, it’s as if even glDeleteTextures doesn’t work… I mean, shouldn’t at least the first indices that were later deleted be returned by glGenTextures ?

To be fair, even if this solution were to work :

r-jaoui:

In the meantime there is still another idea I could try, so just one more question : if a specific texture is generated and the a call to glTexImage2D is made, the texture is used and at a later date I want to “override” this texture with another (possibly of a different size) with glTexImage2D, would that work or is it not allowed ? (issue is that although glDeleteTextures are called, the indices still go up which makes me think that at least this implementation doesn’t return the freed indices first). The solution for me would then be to keep a record of the used textures and if possible, reuse these ones instead of creating new unneccessary ids and so virtually the number of actual texture used would simply be the number of concurrent used textures (in my case 2 instead of 400 ^^).

I’d still not like it since basically this means accepting a certain amount of memory leak, which for a rendering engine is unacceptable for me. The issue is that this is made so that the user can define as many images as he wants and wherever he wants.

GClements · March 24, 2021, 3:54pm

glDeleteTextures allows the implementation to re-use those names; it doesn’t require it.

Midnightas · March 24, 2021, 5:23pm

JOGL is only a Java wrapper around the OpenGL interface. It is not in itself an OpenGL implementation.

r-jaoui · March 24, 2021, 5:32pm

So should textures in JOGL behave the same as in C for example ? Cause for instance in Java the memory is managed by the Garbage Collector and I read that actually OpenGL textures and buffers are stored in JOGL in the main heap and not the Java heap, so outside of the scope of the GC.

I mean, if I wrote this section of code (that I can share if necessary, it is only a bit long) in C would the result be exactly the same or would it behave differently ?

If you need my code for Image management I can share it

Midnightas · March 24, 2021, 6:47pm

The OpenGL implementation uses the main heap, so it is outside of the Java garbage collector. Since the implementation does not care what it is called from, you won’t see any difference. This isn’t something that you can really fix, because the implementation is out of your control.

r-jaoui · March 24, 2021, 7:09pm

Hm okay, so it should be the same in JOGL as in C if I understand you correctly… But then is my implementation of images problematic ? How could an image be stored like this in OpenGL, so that they can be created and used “on the fly” and not be statically loaded ? I mean I just don’t know how I would implement it if not by a texture, I’ve already tried with a BufferedImage but due to the conversions from Java to OpenGL objects this is very slow and it uses about the same amount of textures anyways…

And I’ve also been stuck on this issue for about 2 months now, tried about 3 different image architectures, this was the first relatively fast one, and that worked perfectly, this is the only issue left. :’(

Just in case this could help find if I did something wrong.

My current image implementation

This is SImage.java

package lib;

import java.awt.image.BufferedImage;
import java.awt.image.DataBufferInt;
import java.io.File;
import java.io.IOException;
import java.nio.IntBuffer;

import javax.imageio.ImageIO;

import com.jogamp.opengl.GL;
import com.jogamp.opengl.GL2;

public class SImage {
	SApplet context;

	private boolean initialized;
	private int texture;
	
	public int pixels[];
	
	public int width;
	public int height;

	protected SImage(SApplet context, int width, int height, IntBuffer pixelRaster) {
		this.initialized = false;
		this.width = width;
		this.height = height;
		pixels = new int[width * height];
		this.context = context;
		init(pixelRaster);
	}
	
	protected SImage(SApplet context, int width, int height) {
		this.initialized = false;
		this.width = width;
		this.height = height;
		pixels = new int[width * height];
		this.context = context;
		init();
	}
	
	protected SImage(SApplet context) {
		this.initialized = false;
		width = 0;
		height = 0;
		this.context = context;
	}
	
	protected SImage(SApplet context, int texture) {
		this.initialized = false;
		width = 0;
		height = 0;
		this.context = context;
		init();
		loadTexture(texture);
	}
	
	@Override
	public void finalize() {
		try {super.finalize();}
		catch (Throwable e) {e.printStackTrace();}
		
		System.out.println("DELETING TEXTURE ID : ["+texture+"]");
		
		IntBuffer b = IntBuffer.allocate(1);
		b.put(this.texture);
		this.context.gl.glDeleteTextures(1, b);
	}

	protected void loadTexture(int tex) {
		if(!this.initialized) init();
		
		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, tex);
		IntBuffer buff = IntBuffer.allocate(1);
		this.context.gl.glGetTexLevelParameteriv(GL.GL_TEXTURE_2D, 0, GL2.GL_TEXTURE_WIDTH, buff);
		width = buff.get(0);
		
		buff.clear();
		this.context.gl.glGetTexLevelParameteriv(GL.GL_TEXTURE_2D, 0, GL2.GL_TEXTURE_HEIGHT, buff);
		height = buff.get(0);
		
		IntBuffer pixelRaster = IntBuffer.allocate(width * height);
		this.context.gl.glGetTexImage(GL.GL_TEXTURE_2D, 0, GL2.GL_BGRA, GL.GL_UNSIGNED_BYTE, pixelRaster);
		pixels = pixelRaster.array();
		
		updatePixels();
		
		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, 0);
	}

	protected void init(IntBuffer pixelRaster) {
		IntBuffer tex = IntBuffer.allocate(1);
		
		this.context.gl.glGenTextures(1, tex);
		texture = tex.get(0);
		System.out.println("USING TEXTURE ID (init wt buff) : ["+texture+"]");
		
		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, texture);
		
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MIN_FILTER, GL.GL_LINEAR);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MAG_FILTER, GL.GL_LINEAR);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL2.GL_TEXTURE_BASE_LEVEL, 0);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL2.GL_TEXTURE_MAX_LEVEL, 0);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_S, GL2.GL_CLAMP);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_T, GL2.GL_CLAMP);
	    
		this.context.gl.glTexImage2D(GL.GL_TEXTURE_2D, 0, GL2.GL_RGBA8, width, height, 0, GL2.GL_BGRA, GL2.GL_UNSIGNED_BYTE, pixelRaster);
		
		this.initialized = true;
		
		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, 0);
	}

	protected void init() {
		IntBuffer dat = IntBuffer.allocate(width * height);
		IntBuffer tex = IntBuffer.allocate(1);
		
		this.context.gl.glGenTextures(1, tex);
		texture = tex.get(0);
		System.out.println("USING TEXTURE ID (init no buff) : ["+texture+"]");
		
		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, texture);
		
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MIN_FILTER, GL.GL_LINEAR);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MAG_FILTER, GL.GL_LINEAR);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL2.GL_TEXTURE_BASE_LEVEL, 0);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL2.GL_TEXTURE_MAX_LEVEL, 0);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_S, GL2.GL_CLAMP);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_T, GL2.GL_CLAMP);
	    
		this.context.gl.glTexImage2D(GL.GL_TEXTURE_2D, 0, GL2.GL_RGBA8, width, height, 0, GL2.GL_BGRA, GL.GL_UNSIGNED_BYTE, dat);
		
		this.initialized = true;
		
		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, 0);
	}

	protected int getTexture() {
		if(!this.initialized) init();
		return this.texture;
	}

	public void resize(int width, int height) {
		if(!this.initialized) init();
		this.width = Math.max(1, width);
		this.height = Math.max(1, height);
		pixels = new int[this.width * this.height];

		IntBuffer b = IntBuffer.allocate(1);
		b.put(texture);
		this.context.gl.glDeleteTextures(0, b);
		
		init();
	}

	public void loadPixels() {
		if(!this.initialized) init();

		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, texture);
		IntBuffer pixelRaster = IntBuffer.allocate(width * height);
		this.context.gl.glGetTexImage(GL.GL_TEXTURE_2D, 0, GL2.GL_BGRA, GL.GL_UNSIGNED_BYTE, pixelRaster);
		pixels = pixelRaster.array();
		
		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, 0);
	}

	public void updatePixels() {
		if(!this.initialized) init();
		
		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, texture);
		this.context.gl.glTexImage2D(GL.GL_TEXTURE_2D, 0, GL.GL_RGBA8, width, height, 0, GL2.GL_BGRA, GL.GL_UNSIGNED_BYTE, IntBuffer.wrap(pixels));
		
		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, 0);
	}

	public void save(String filename) {
		if(!this.initialized) init();
		
		loadPixels();
		
		BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
		final int[] a = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();
		System.arraycopy(pixels, 0, a, 0, pixels.length);
		
		File file = new File(filename);
		String[] sections = filename.split("\\.");
		BufferedImage img;
		
		if(sections[sections.length - 1].equals("jpg")  ||
		   sections[sections.length - 1].equals("jpeg") ||
		   sections[sections.length - 1].equals("JPG")  ||
		   sections[sections.length - 1].equals("JPEG")) {
			img = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
			img.createGraphics().drawImage(image, 0, 0, null);
		}
		else if(sections[sections.length - 1].equals("bmp")  ||
		        sections[sections.length - 1].equals("BMP")){
			img = new BufferedImage(width, height, BufferedImage.TYPE_3BYTE_BGR);
			img.createGraphics().drawImage(image, 0, 0, null);
		}
		else if(sections[sections.length - 1].equals("wbmp")  ||
		        sections[sections.length - 1].equals("WBMP")){
			img = new BufferedImage(width, height, BufferedImage.TYPE_BYTE_BINARY);
			img.createGraphics().drawImage(image, 0, 0, null);
		}
		else {
			img = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
			img.createGraphics().drawImage(image, 0, 0, null);
		}
		img.createGraphics().finalize();
		try {
			ImageIO.write(img, sections[sections.length - 1], file);
		} catch (IOException e) {
			e.printStackTrace();
		}
	}

	public SImage copy() {
		if(!this.initialized) init();
		
		return new SImage(this.context, texture);
	}

	private float getXTex(SImage img, float x) {
		return (x+0.5f) / (float) img.width;
	}
	
	private float getYTex(SImage img, float y) {
		return (y + 0.5f) / (float) img.height;
	}
	
	private float getXScreen(float x) {
		return 2f * (x+0.5f) / (float) context.width - 1f;
	}
	
	private float getYScreen(float y) {
		return 2f * (y + 0.5f) / (float) context.height - 1f;
	}

	public void copy(float sx, float sy, float sw, float sh, float dx, float dy, float dw, float dh) {
		if(!this.initialized) init();
		
		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, texture);
		IntBuffer pixelRaster = IntBuffer.allocate(width * height);
		this.context.gl.glGetTexImage(GL.GL_TEXTURE_2D, 0, GL2.GL_BGRA, GL.GL_UNSIGNED_BYTE, pixelRaster);
		
		IntBuffer texBuffer = IntBuffer.allocate(1);
		this.context.gl.glGenTextures(1, texBuffer);

		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, texBuffer.get(0));
		
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MIN_FILTER, GL.GL_LINEAR);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MAG_FILTER, GL.GL_LINEAR);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL2.GL_TEXTURE_BASE_LEVEL, 0);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL2.GL_TEXTURE_MAX_LEVEL, 0);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_S, GL2.GL_CLAMP);
		this.context.gl.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_T, GL2.GL_CLAMP);
	    
		this.context.gl.glTexImage2D(GL.GL_TEXTURE_2D, 0, GL2.GL_RGBA8, width, height, 0, GL2.GL_BGRA, GL.GL_UNSIGNED_BYTE, pixelRaster);
		
		//context.getTextureRenderer() RETURNS THE ID OF A STATIC FBO, GENERATED ON init() THAT SHOULD BE USED FOR IMAGE MANIPULATION
		
		this.context.gl.glBindFramebuffer(GL.GL_FRAMEBUFFER, context.getTextureRenderer());
		this.context.gl.glFramebufferTexture2D(GL.GL_FRAMEBUFFER, GL2.GL_COLOR_ATTACHMENT0, GL.GL_TEXTURE_2D, texture, 0);
		this.context.gl.glDrawBuffer(GL2.GL_COLOR_ATTACHMENT0);
		
		this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, texBuffer.get(0));

		float sx0 = getXTex(this, sx);
		float sy0 = getYTex(this, sy);
		float sx1 = getXTex(this, sx + sw);
		float sy1 = getYTex(this, sy + sh);
		
		float dx0 = getXScreen(dx);
		float dy0 = getYScreen(dy);
		float dx1 = getXScreen(dx + dw);
		float dy1 = getYScreen(dy + dh);
		
		this.context.gl.glColor4f(1f, 1f, 1f, 1f);
		
		this.context.gl.glBegin(GL2.GL_QUADS);
		this.context.gl.glTexCoord2f(sx0, sy0);
		this.context.gl.glVertex2f(dx0, dy0);
		this.context.gl.glTexCoord2f(sx0, sy1);
		this.context.gl.glVertex2f(dx0, dy1);
		this.context.gl.glTexCoord2f(sx1, sy1);
		this.context.gl.glVertex2f(dx1, dy1);
		this.context.gl.glTexCoord2f(sx1, sy0);
		this.context.gl.glVertex2f(dx1, dy0);
		this.context.gl.glEnd();
		this.context.gl.glFlush();
		
		this.context.gl.glFramebufferTexture2D(GL.GL_FRAMEBUFFER, GL2.GL_COLOR_ATTACHMENT0, GL.GL_TEXTURE_2D, 0, 0);
		this.context.gl.glBindTexture(GL2.GL_TEXTURE_2D, 0);
		this.context.gl.glDeleteTextures(1, texBuffer);

		this.context.gl.glBindFramebuffer(GL.GL_FRAMEBUFFER, 0);
		this.context.gl.glDrawBuffer(GL.GL_FRONT_AND_BACK);
	}

	public void copy(SImage src, float sx, float sy, float sw, float sh, float dx, float dy, float dw, float dh) {
		if(!this.initialized) init();
		
		if(src.getTexture() == texture) {
			copy(sx, sy, sw, sh, dx, dy, dw, dh);
		}
		else {
			//context.getTextureRenderer() RETURNS THE ID OF A STATIC FBO, GENERATED ON init() THAT SHOULD BE USED FOR IMAGE MANIPULATION
			
			this.context.gl.glBindFramebuffer(GL.GL_FRAMEBUFFER, context.getTextureRenderer());
			this.context.gl.glFramebufferTexture2D(GL.GL_FRAMEBUFFER, GL2.GL_COLOR_ATTACHMENT0, GL.GL_TEXTURE_2D, texture, 0);
			this.context.gl.glDrawBuffer(GL2.GL_COLOR_ATTACHMENT0);
			
			this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, src.getTexture());
	
			float sx0 = getXTex(src, sx);
			float sy0 = getYTex(src, sy);
			float sx1 = getXTex(src, sx + sw);
			float sy1 = getYTex(src, sy + sh);
			
			float dx0 = getXScreen(dx);
			float dy0 = getYScreen(dy);
			float dx1 = getXScreen(dx + dw);
			float dy1 = getYScreen(dy + dh);
			
			this.context.gl.glColor4f(1f, 1f, 1f, 1f);
			
			this.context.gl.glBegin(GL2.GL_QUADS);
			this.context.gl.glTexCoord2f(sx0, sy0);
			this.context.gl.glVertex2f(dx0, dy0);
			this.context.gl.glTexCoord2f(sx0, sy1);
			this.context.gl.glVertex2f(dx0, dy1);
			this.context.gl.glTexCoord2f(sx1, sy1);
			this.context.gl.glVertex2f(dx1, dy1);
			this.context.gl.glTexCoord2f(sx1, sy0);
			this.context.gl.glVertex2f(dx1, dy0);
			this.context.gl.glEnd();
			this.context.gl.glFlush();
			
			this.context.gl.glBindTexture(GL2.GL_TEXTURE_2D, 0);

			this.context.gl.glFramebufferTexture2D(GL.GL_FRAMEBUFFER, GL2.GL_COLOR_ATTACHMENT0, GL.GL_TEXTURE_2D, 0, 0);
			this.context.gl.glBindFramebuffer(GL.GL_FRAMEBUFFER, 0);
			this.context.gl.glDrawBuffer(GL.GL_FRONT_AND_BACK);
		}
	}

	public SImage get() {
		return this;
	}

	public int get(int x, int y) {
		if(x >= 0 && x < width && y >= 0 && y < height) {
			this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, texture);
			IntBuffer pixelRaster = IntBuffer.allocate(width * height);
			this.context.gl.glGetTexImage(GL.GL_TEXTURE_2D, 0, GL2.GL_BGRA, GL.GL_UNSIGNED_BYTE, pixelRaster);
			this.context.gl.glBindTexture(GL.GL_TEXTURE_2D, 0);
			return pixelRaster.get(x + y * width);
		}
		else return SGraphics.color(0, 0, 0);
	}

	public SImage get(int x, int y, int w, int h) {
		IntBuffer initialRaster = IntBuffer.allocate(w * h);
		
		SImage copy = new SImage(this.context, w, h, initialRaster);
		
		float actual_x0 = (float) Math.max(x, 0);
		float actual_y0 = (float) Math.max(y, 0);
		float actual_x1 = (float) Math.min(x + w, width);
		float actual_y1 = (float) Math.min(y + h, height);
		float actual_w = actual_x1 - actual_x0;
		float actual_h = actual_y1 - actual_y0;
		
		copy.copy(this, actual_x0, actual_y0, actual_w, actual_h, actual_x0 - x, actual_y0 - y, actual_w, actual_h);
		
		return copy;
	}

	public void set(int x, int y, int c) {
		int pixel[] = {c};
		this.context.gl.glTexSubImage2D(GL.GL_TEXTURE_2D, 0, x, y, 1, 1, GL.GL_BGRA, GL.GL_UNSIGNED_BYTE, IntBuffer.wrap(pixel));
	}

	public void set(int x, int y, SImage img) {
		copy(img, 0, 0, img.width, img.height, x, y, img.width, img.height);
	}

	protected BufferedImage toImage() {
		loadPixels();
		
		BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
		final int[] a = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();
		System.arraycopy(pixels, 0, a, 0, pixels.length);
		
		return image;
	}
}

This is the code that is run at each image (so around 60 times a second)

@Override
public void draw() {
    background(255);

    SImage im = createImage(1280, 720);
		
    stroke(0);
    strokeWeight(2f);
    noFill(255, 0, 0);

    rect(mouseX, mouseY, 20, 20);
}

And this is the createImage() method

public SImage createImage(int width, int height) {
	return new SImage(this, width, height);
}

Thank you very much for your help though !

GClements · March 25, 2021, 6:42am

Yes.

Textures and buffers are typically stored in video memory.

They can’t be managed by Java’s GC; or by any allocator with delayed finalisation, for that matter. A common mistake when trying to force OOP paradigms on OpenGL is to wrap OpenGL objects in a class and call glDelete* from the destructor. This falls down when the destructor gets called with the wrong context bound (or with no context bound, e.g. at program termination). You can just about get away with it in C++ due to immediate finalisation (i.e. you know exactly when the destructor is called), but it requires the programmer to be paying attention to object lifetimes and in practice tends to cause more problems than it solves.

The other side of the coin is that any objects will be deleted by the implementation if the context to which they belong is destroyed. Unless you can guarantee the sequencing of opererations so that a context isn’t deleted until after the objects which belong to it have been deleted, the underlying OpenGL objects may simply vanish while their wrappers still exist.

In short, remember that your wrappers don’t “own” the OpenGL objects which they create. The implementation owns them; the wrappers just have references to them.

mhagain · March 25, 2021, 9:43am

Just to be absolutely clear to you - OpenGL is not software. It’s not a software library, it’s an interface to your graphics hardware; when you make an OpenGL call, what happens is that the OpenGL interface instructs your video card device driver to send the necessary commands to your video card, and your video card does the actual work.

Likewise OpenGL objects are typically stored in video card memory. They may have backing copies of them also stored in CPU memory, but this isn’t required by the OpenGL specification.

Likewise, OpenGL is not in any way OO-aware. Treat it as a C API, with one big global namespace and all objects and state being global to the current context. OO wrappers may try to hide that, but it’s just syntactic sugar, and you really do need to be aware of what it’s actually like to avoid getting bitten by it. I’d actually be of the opinion that OO “prettiness” is more a hindrance than a help in this respect; better that you get exposed to what things are actually like so that you can write code that behaves correctly.

So when you’re writing code to manage OpenGL objects you really do need to bear all of this in mind.

r-jaoui · March 25, 2021, 10:10am

Wow, thank you both for all of these details

Well I guess all that is left for me to do is to find a data structure for storing images that is more forgiving in terms of memory usage.

EDIT : I had a thought… Since I already have a pixels array, could I store the image only in the pixel array and display it with the glDrawPixel() function and not using any Texture whatsoever ? In this way the image itself would technically be stored in Java memory and the GC would have full control of their destruction.

Do you think that this would work better in terms of memory management ? (The only issues would be the hard functions such as the copy function that should be able to quickly draw another image on top with possibly scaling factors, as well as the fact that since none of these functions would be parallelized and there is a risk that they would run slow). I’d also need to actually render primitives onto an image, with multisampling and so on, so I guess that this isn’t really possible.

GClements · March 25, 2021, 12:24pm

If you’re going to use the same data multiple times, glDrawPixels is inefficient as you’re transferring the data to the video card each time.

But if you’re going to take that approach, there probably isn’t much point in using OpenGL at all. You’ll probably get better results with Java’s native GUI functionality (whatever that is now; the last time I used Java, Swing was new and AWT was still common).

r-jaoui · March 25, 2021, 12:40pm

Yeah I know Swing pretty well, but this wouldn’t work for everything since at some points I’ll need to draw OpenGL primitives onto images, so using native Swing wouldn’t work or would be terribly slow (for instance, rendering a texture onto a random quad is virtually impossible in a correct amount of time in Swing as far as I’m concerned).

Furthermore, in this step I only need basic image functions (copy, get, set) but later on I’d need to do everything that OpenGL can on an image (which also includes GLSL shaders, stencil testing…), so I need OpenGL (shaders and stencil testing are actually, for now at least, the only reason I’m using OpenGL. Other than that this is purely 2D so I don’t have any need for 3D rendering… But the 2D renderer needs to be really fast, work off-screen and be able to run shaders).

And other than that, if I can’t find another way to mitigate the memory issue, I would need to add the fact that all images need to be created statically basically… Which in itself is a huge specification to impose to the user, and a huge drawback for this project…

If I were to draw to a FrameBuffer and want to gather pixel data to store it at a later date, would I need to bind a texture to this buffer first (otherwise there would be no data on which to draw ?).

GClements · March 25, 2021, 12:50pm

If you need to use shaders, then image data realistically needs to be stored in textures.

A FBO is just a state container. It needs at least one attachment in order to be useful. Attachments can be textures or renderbuffers. If you’re going to use the rendered data as a source for subsequent rendering operations, use a texture.

r-jaoui · March 25, 2021, 2:08pm

Yeah so I pretty much need textures, and then I have no option other than to accept that there will be leaks… Oh well

r-jaoui · March 25, 2021, 5:34pm

Alright so I guess I’m dumb…

I retried with the manual close function (without the GC-based finalize() method) and using the initally generated IntBuffer as the glDeleteTextures target… And I have no leak whatsoever and the memory consumption seems constant (higher at higher FPS, which can be expected since textures are loaded and unloaded so fast that memory has more “overlap”).

So I guess that the issue was that the GC was so slow to call the finalize method that at this point a lot of textures weren’t discarded and it looked like a leak… where it was just that tens of textures were not used and deleted too late by the GC (I bencharked a single of these textures to be about 30MB, so even as few as 30 simultaneous would be around 1GB, and given that the GC’s destructions could sometimes be of 50 at a time… A few GB is to be expected).

To be fair, I actually don’t even think that the glDeleteTextures call worked since when replacing the IntBuffer in finalize() with the original one, I got this error (I’m still curious on what it means) :

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000000715475d9, pid=28448, tid=0x0000000000006d84
#
# JRE version: Java(TM) SE Runtime Environment (8.0_151-b12) (build 1.8.0_151-b12)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.151-b12 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C  [nvoglv64.DLL+0xa175d9]
#
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# D:\Programmation\Workspaces\Workspace 8 - Processing\SGLRL\hs_err_pid28448.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

But then, if I replace the finalize() call with a close() function, my memory consumption doesn’t get above a few hundred MB (expected for such a large number of images) and above all, DOESN’T INCRESE (which was the case even with a close() function if the glDeleteTextures() target wasn’t the actual initial IntBuffer).

Sooo yup, thank you all for your help, I think I finally managed to get this working

system · September 24, 2021, 5:34pm

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.