For the past few days, I had an idea to stencil out any shape, then render something else only inside the stencil, so I researched a bit. The game I’m developing for, Minecraft does NOT have a stencil buffer for the framebuffer. I’m coding a mod for FabricMC. After finding this out, I decided to start asking some people with OpenGL knowledge what are some alternatives I can use instead of a stencil buffer. Everyone had a similar response, which was to use a shader. I am coding in java, and I had to learn the fundamentals of GLSL in a short period of time. To be clear, I do not know hardly anything about shaders, and after hours of searching, there is little to no information about my issue that I could find.
Allow me to explain exactly what I’d like the final result to be.
I would like to be able to render ANY shape, then use that shape as a mask, so that I can render anything I want, but it will only show what I’ve rendered if it’s inside the mask. For now, I’d only like to render a simple circle, and use that circle as a mask.
My ultimate goal would be rendering a gradient rectangle on the CPU, and masking the rectangle into a circle shape, and rendering only the circle.
This is the fragment shader I have coded: [GLSL]
#version 460
layout(origin_upper_left) in vec4 gl_FragCoord;
in vec4 gl_Color;
in vec2 texCoord;
uniform sampler2D MaskSampler;
uniform vec2 center;
uniform float radius, scale;
out vec4 fragColor;
const void main() {
vec4 col = texture(MaskSampler, texCoord);
vec2 fragPos = gl_FragCoord.xy;
float dist = distance(fragPos, center) * scale;
if(dist > radius) discard;
fragColor = col;
}
This is the JSON file for the shader: [JSON]
{
"blend": {
"func": "add",
"srcrgb": "srcalpha",
"dstrgb": "1-srcalpha"
},
"vertex": "sobel",
"fragment": "ellipse",
"attributes": ["Position", "Color"],
"samplers": [{"name": "MaskSampler"}],
"uniforms": [
{
"name": "ProjMat",
"type": "matrix4x4",
"count": 16,
"values": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0]
},
{
"name": "center",
"type": "float",
"count": 2,
"values": []
},
{
"name": "radius",
"type": "float",
"count": 1,
"values": []
},
{
"name": "scale",
"type": "float",
"count": 1,
"values": []
}
]
}
This is my actual code implementation of applying the shader: [Java]
private static final ShaderInstance ELLIPSE_SHADER = new ShaderInstance("ellipse", ELLIPSE_FSH).builder()
.addUniform("center", "float", 2)
.addUniform("radius", "float", 1)
.addUniform("scale", "float", 1)
.build();
@SneakyThrows
public static void renderCircleGradient(MatrixStack stack, float originX, float originY, float rad, Color... colors) {
final Window w = client.getWindow();
final float scaledX = originX / w.getScaledWidth() * w.getFramebufferWidth(),
scaledY = originY / w.getScaledHeight() * w.getFramebufferHeight(),
scaledRad = rad / w.getScaledWidth() * w.getFramebufferWidth();
setupRender();
stack.push();
ELLIPSE_SHADER
.setUniform("center", scaledX, scaledY)
.setUniform("radius", scaledRad)
.setUniform("scale", (float) w.getScaleFactor())
.apply(); // This sets the current shader to this one.
renderCustomGradient(stack.peek().getPositionMatrix(), colors, scaledX - scaledRad, scaledY - scaledRad, scaledX + scaledRad, scaledY + scaledRad);
stack.pop();
endRender();
}
The intended behavior of this shader is to hook it onto a render layer, and then I’d like to apply the shader (which is the mask) then, on the CPU I’d like to be able to render anything I want and it would only show up inside the mask.
The current behavior of the shader renders the mask correctly, however I cannot figure out how to allow rendering from the CPU to appear within the mask. All the shader does is render a black circle at the specified position.
Some potential problems that could be causing this unintended behavior:
- I do not know how to correctly configure the sampler2D (MaskSampler), and I cannot find any information on this topic that’s related to Minecraft, or even Java.
- I do not know how to set up a mask for the shader json file.
- I am rendering the shader incorrectly.
These are some solutions that were recommended to me:
- Draw my gradient image on the CPU, create an OpenGL texture from it, and then bind the texture to a sampler uniform so I read it from my shader.
- Make my own post processing shader, draw something to a mask framebuffer and use that mask framebuffer to blit another framebuffer into the main buffer.
Here’s what I’d like help on:
- Getting the second potential solution to work. Learning how masks work, learning what the hell a blit is; masking w/ a framebuffer.
- Figuring out how to bind a texture from a non-image source. (Preferrably from a section of pixels on the screen) and then passing it into my shader.
- Fixing my GLSL shader so that it renders anything inside the mask.
- Finding any alternative, potentially easier methods of accomplishing my goal.