Need help Using a pre-made Stencil

I’ve been googling for 4 hours and decided I should probably make a post. I have a pre-made 0/1 stencil mask I want to use to decide if I should render the pixel or not. The mask is not just a simple rect.

questions:
Is a depth-stencil image one image or are there separate depth and stencil components like the name suggests?

Is there a specific way to upload to the stencil component of the depth-stencil VkImage(if there are separate components)? Is it similar to uploading data to a texture VkImage?

After I upload my premade mask to the image, do I have to do this every frame? Or do I just have to set the stencil load and store ops in the renderpass depthattachment to load and store (or dont care?)so I don’t have to keep uploading it every frame?

If I fill out the stencil test/state info in the vkgraphicspipeline is that enough (in addition to the above) to get it to not render those pixels with 0 in the corresponding stencil location?

Taking a look to the depth stencil formats, you see the depth and the stencil buffers are together.

No, there is no specific way. you can copy data to it just like any other image.

If you want to have the stencil done on every image, you have to fill the buffer on every frame, unless you don’t change the depth-stencil buffer, in such case, uploading it just once is fine. Just make sure you don’t destroy it when begining a new frame, if you’re not changing it.

You have to fill all the necesary information in your pipeline. The pipeline defines how your render will be done, if you don’t write it properly, don’t expect it to work correctly.

[QUOTE=SantiagoAlberto;42880]Taking a look to the depth stencil formats, you see the depth and the stencil buffers are together.

You have to fill all the necesary information in your pipeline. The pipeline defines how your render will be done, if you don’t write it properly, don’t expect it to work correctly.[/QUOTE]

Thanks for the reply Santiago! Looking at the possible formats, I see that the format I’ll need is probably VK_FORMAT_D32_SFLOAT_S8_UINT meaning for each pixel there is a {float, uint8_t} pair. So I guess when I do my uploading I need to make sure I stride by the size of float+uint8 (5 bytes) and offset by 4 bytes to skip the float.

For more details I am doing a VR renderer and need to use a radial density mask(https://gifyu.com/images/ValveRDM2.jpg) to not render pixels that wont be seen through the lens or are off each eye’s view axis enough that they shouldn’t be renderered at full res since our perif vision falls off in detail the more off angle you go.

I will be uploading once and keeping the stencil around since its a static mask that I need so the forward render pass doesnt draw if the lsb bit is 0. To keep it around I guess I need to make sure the renderpass describing the forward pass needs to have:
depthAttachment.stencilLoadOp = VK_ATTACHMENT_LOAD_OP_LOAD;//previous contents preserved (i guess clearing wont affect it)
depthAttachment.stencilStoreOp = VK_ATTACHMENT_STORE_OP_STORE;//keep around for next time

my clear values in the render pass begin info need to not effect the stencil, I assume this would imply I just set 1.f for the depth as usual but nothing for the stencil = {1.f}

other than my stencil test is configured as:

depthStencil.stencilTestEnable = VK_TRUE;
VkStencilOpState stencilOpState = {}; 
    uint8_t mask = 0x1;
stencilOpState.compareMask = mask;//AND'd with reference val to get final compare val to test against stencil val
stencilOpState.reference = mask;
stencilOpState.writeMask = mask;//alternatively set to 0 to save a write operation since this stencil is static
stencilOpState.compareOp = VK_COMPARE_OP_EQUAL; //render frag if true
//what to do with stored stencil val in these events
stencilOpState.depthFailOp = VK_STENCIL_OP_KEEP;// frag fails depth
stencilOpState.failOp = VK_STENCIL_OP_KEEP;//stencil fails ref
stencilOpState.passOp = VK_STENCIL_OP_KEEP;//stencil passes ref
depthStencil.front = stencilOpState;

also aspect masks need to be OR’d in to the depth aspect to get the image view to be correct
aspectFlags = VK_IMAGE_ASPECT_DEPTH_BIT | VK_IMAGE_ASPECT_STENCIL_BIT;

and when the final transition occurs for the depth stencil init, there needs to read/write enable and early frag tests
else if (oldLayout == VK_IMAGE_LAYOUT_UNDEFINED && newLayout == VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL) {
barrier.srcAccessMask = 0;
barrier.dstAccessMask = VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT;
sourceStage = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
destinationStage = VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT;

So I guess when I do my uploading I need to make sure I stride by the size of float+uint8 (5 bytes) and offset by 4 bytes to skip the float.

Please look at the format as it is described in the Vulkan specification. It is very explicit about it, and it is explicitly not that.

Note that the way depth/stencil formats are handled in Vulkan is unusual. When copying data from buffers into such images, your copy goes to the “depth aspect” or “stencil aspect” of those formats. You aren’t allowed to copy into both at the same time.

The stencil aspect is always “tightly packed” in 8-bits per stencil value. So the stencil and depth components in the source buffer should be in separate locations.

So there is no need to do any offsetting when using vkCmdCopyBufferToImage. You can just have an appropriate array of 8-bit values for your stencil.

Thanks for heads up Alfonse.
“VK_FORMAT_D32_SFLOAT_S8_UINT specifies a two-component format that has 32 signed float bits in the depth component and 8 unsigned integer bits in the stencil component. There are optionally: 24-bits that are unused.”

So then it’s {float,uint32_t} for each depth-stencil pixel, and the upper 24msb bits are unused in the stencil component.

After reading vkCmdCopyBufferToImage and VkBufferImageCopy, I didn’t see how an copy an array of {uint32_t} to and image of {float, uint32_t} so I guess I’ll pad by source buffer with floats so I don’t have fiddle with the command settings. I’ll make an array of size widthheight with {float, uint32_t} struct elements where the uint32_t’s are sourced from my premade stencil. the depth floats don’t need to be set since they are cleared anyway. cast the array to uint8_t and put it in a buffer and put the buffer in the depth stencil image without any guesswork.

I didn’t see how an copy an array of {uint32_t} to and image of {float, uint32_t}

You don’t. As it says in the spec:

data copied to or from the stencil aspect of any depth/stencil format is tightly packed with one VK_FORMAT_S8_UINT value per texel.

So like I said, your source data is an array of 8-bit stencil values.

[QUOTE=Alfonse Reinheart;42885]You don’t. As it says in the spec:

So like I said, your source data is an array of 8-bit stencil values.[/QUOTE]

My bad Alfonse, didn’t see your second response explaining the particulars of copying a buffer into a depthstencil image, I must have been writing my response to your comment about my misunderstanding of the float padding.

Got it, so I take a width*height array of my pre-calculated uint8_t stencil values put them in a buffer and when I copy to the depth stencil image the command configuration is:

    VkBufferImageCopy region = {};
    region.bufferOffset = 0;
    region.bufferRowLength = 0;
    region.bufferImageHeight = 0;
    region.imageSubresource.aspectMask = VK_IMAGE_ASPECT_STENCIL_BIT
    region.imageSubresource.mipLevel = 0;
    region.imageSubresource.baseArrayLayer = 0;
    region.imageSubresource.layerCount = 1;
    region.imageOffset = {0, 0, 0};
    region.imageExtent = {
        width,
        height,
        1
    };

    vkCmdCopyBufferToImage(commandBuffer, buffer, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &region);

I should be good to go now, thanks for your help, really appreciate it!