It seems I have a kind of standard problem.
I need to pass data from one rendering stage to next one.
The most using tool for that is a texture buffer.
And I did that so in scope of implementation “Shadow Map Silhouette Revectorization (SMSR)”.
There is format of data:

It is so difficult to improve it and support.
Decoding such data in the second pass is not human friendly at all.
I think there are some variants of passing data different from two float channels.
I know I can use SSBO with custom structures, but it waist many memory for packing holes.

Could you advice some variants to use more friendly data structures?

Then you can create a function to unpack the data at an index as long as you know the offset of each member in your structure you need to access, an example could be

This is how it can be done if you would like to pack the data and unpack the data and makes it more human friendly when using it.

I need to pass data from one rendering stage to next one.

One thing that you haven’t really mentioned is what type of data you need to pass or why, you say that you did so in scope of implementation of “Shadow Map Silhouette Revectorization (SMSR)” however are you using this for shadow mapping?

Ultimately if you are trying to pass values that are smaller than 4 floats and are needed by each pixel then storing the data in a texture would be the best most human friendly way as a simple imageLoad()/texelFetch()/texture() would do the trick for fetching the memory. if the data is a lot more complex and you need to save on memory you can use the method I showed you above with a storage buffer. If memory isn’t to much of an issue then you can just pass the structure directly instead of packing/unpacking but make sure to follow alignment rules.

Could you advice some variants to use more friendly data structures?

For what? what rendering technique are you trying to do, if its the shadow map you are generating then definitely a texture, if its passing pixel specific data a texture also works, if its passing say, nodes in a tree then a shader storage buffer. If you need more information then be a little more specific for what you need the data structure for as I haven’t fully understood what you are trying to do.

GLSL is very similar to C so treat it as such, use structs to abstract data and just treat memory as normal memory you would have in your host program. If you need something more specific let me know.

EDIT:
I forgot to mention that for storing data in an image you should use imageStore(), if you want to manipulate/fill a storage buffer with memory then make sure you have enough space allocated from host and you can access it directly as an array.

First pass. In screen space (fragment shader) detects shadow discontinuities and encodes it in two channels of texture red and green (see the table above): Red encodes horizontal discontinuities and Green vertical. 0 - no discontinuity, 0.5 - discontinuity in one diration, 0.75 - in both directions and 1 - in second direction.
Thus I have two channels with four possible values per each. 16 - possible values.

Second pass. Also during rasterization (fragment shader) I calculate shadow border using data with discontinuities from first pass.

I think the best way to explain that is a revectorization shader as is. You can see the hell of dot products and other difficult manipulations:

#version 430
#define MAX_SEARCH 4
#define NO_DICS vec2(0)
#define EPS .01
const int NUM_CASCADES = 4; // fix for nvidia, actually it needs to be 3
in vec2 outTexCoord;
out vec4 fragColor;
struct DiscEdge {
int length;
bool closed;
};
uniform sampler2D deferredPosition;
uniform sampler2D deferredWPosition;
uniform sampler2DArray dlShadowMap;
uniform sampler2D discontinuityMap;
uniform float cascadeFarPlanes[NUM_CASCADES];
uniform mat4 dlProjLightViewMatrices[NUM_CASCADES];
uniform mat4 invertedDlProjLightViewMatrices[NUM_CASCADES];
// add dl texel size
uniform mat4 viewMatrix; // todo calc mul with projectionMatrix on cpu side
uniform mat4 projectionMatrix;
vec3 getDicsSample(vec2 basePoint, vec2 texelSize, vec2 offset, int dlMapLayer) {
vec2 newSamplePosition = basePoint + offset * texelSize;
float textDepth = texture(dlShadowMap, vec3(newSamplePosition, dlMapLayer)).r;
vec4 dlViewProjPosition = vec4( vec3(newSamplePosition, textDepth) * 2 - 1, 1);
vec4 newWposition = invertedDlProjLightViewMatrices[dlMapLayer] * dlViewProjPosition;
vec4 buff = projectionMatrix * viewMatrix * newWposition;
vec2 newCreenSpacePosition = (buff.xyz / buff.w * .5 + .5).xy;
return texture(discontinuityMap, newCreenSpacePosition).xyz;
}
bool eq(float value1, float value2) {
return abs(value1 - value2) < EPS;
}
bool eq(vec2 value1, vec2 value2) {
return abs(value1.x - value2.x) < EPS && abs(value1.y - value2.y) < EPS;
}
bool hasDiscBySign(float discValue, float dirSign) {
if (discValue == 0) return false; // no disc
else if (eq(discValue, .75)) return true; // in both dirs
else {
if (dirSign == 1 && discValue == 1) return true;
else if (dirSign == -1 && eq(discValue, .5)) return true;
else return false;
}
}
// works only if dir along one of axises
bool hasDiscToDir(vec2 disc, vec2 dir) {
return (abs(dir.x) == 1)
? hasDiscBySign(disc.x, dir.x)
: hasDiscBySign(disc.y, dir.y);
}
bool isEdge(vec2 baseDisc, vec2 direction, vec2 dicsSample){
float y = direction.x == 0 ? direction.y : -direction.y;
float axisValue = dot(vec2(direction.x, y), vec2(1.)) / 4 + .75;
float edgeNormal = dot(abs(direction.yx), baseDisc);
return eq(edgeNormal, dot(abs(direction.yx), dicsSample))
&& eq(axisValue, dot(abs(direction.xy), dicsSample));
}
DiscEdge findEdge(vec3 baseDisc, vec2 dlProjCoords, vec2 texelSize, vec2 direction, int dlMapLayer) {
if (dot(baseDisc.yx, direction) == 0) {
return DiscEdge(0, false);
}
if(hasDiscToDir(baseDisc.xy, direction)) {
return DiscEdge(0, baseDisc.z == 0);
}
float edgeBaseValue = dot(abs(direction.yx), baseDisc.xy);
for (int i = 1; i < MAX_SEARCH; i++) {
vec3 dicsSample = getDicsSample(dlProjCoords.xy, texelSize, direction * i, dlMapLayer);
// check if value is changes along normal of edge
if(!eq(edgeBaseValue, dot(abs(direction.yx), dicsSample.xy))) {
return DiscEdge(i - 1, false);
}
if(dicsSample.xy == NO_DICS) {
return DiscEdge(i - 1, false);
}
if (isEdge(baseDisc.xy, direction, dicsSample.xy)) {
return DiscEdge(i, dicsSample.z == 0);
}
}
return DiscEdge(MAX_SEARCH - 1, false);
}
// calculating in edge oriented space
float interpolate(vec2 dlProjSampleCoords, DiscEdge from, DiscEdge to, vec2 disc, vec2 edgeAxis, vec2 texelSize) {
bool isHorizontalEdge = edgeAxis.x == 1;
vec2 edgeOrientedSampleCoords = isHorizontalEdge ? dlProjSampleCoords : dlProjSampleCoords.yx;
float edgeLength = min(from.length + to.length + 1.0f, MAX_SEARCH);
float dHigh = 1.0 / edgeLength;
// is here must be only one closed edge: both closed must be calculated separately
if (!from.closed && !to.closed || from.closed && to.closed) {
return 0;
}
float edgeSign = from.closed ? -1.0 : 1.0;
// float edgeSign = to.closed ? 1.0 : -1.0;
float edgeSteps = edgeSign == 1
? min(MAX_SEARCH - 1 - to.length, from.length)
: min(MAX_SEARCH - 1 - from.length, to.length);
float value = edgeSign == 1
? (edgeSteps + edgeOrientedSampleCoords.x) * dHigh
: (edgeSteps + 1 - edgeOrientedSampleCoords.x) * dHigh;
float edgeNormalSign = (dot(edgeAxis.yx, disc) - .75) * 4.;
return edgeNormalSign == 1
? (1 - edgeOrientedSampleCoords.y < value ? 1 : 0)
: (edgeOrientedSampleCoords.y < value ? 1 : 0);
}
void main() {
vec3 disc = texture(discontinuityMap, outTexCoord).xyz;
if (disc.xy == NO_DICS) {
discard;
}
if ((eq(disc.x, .75) && eq(disc.y, 0)) || (eq(disc.y, .75) && eq(disc.x, 0))) {
discard;
}
vec4 position = vec4(texture(deferredPosition, outTexCoord).xyz, 1.0);
vec4 wPosition = vec4(texture(deferredWPosition, outTexCoord).xyz, 1.0);
int dlMapLayer;
for (int i = 0; i < NUM_CASCADES; i++) {
if ( abs(position.z) < cascadeFarPlanes[i] ) {
dlMapLayer = i;
break;
}
}
vec4 dlViewCascadedPosition = dlProjLightViewMatrices[dlMapLayer] * wPosition;
ivec2 shadowMapSize = textureSize(dlShadowMap, 0).xy;
vec2 texelSize = 1.f / shadowMapSize;
vec2 dlProjCoords = dlViewCascadedPosition.xy;
dlProjCoords = dlProjCoords * 0.5 + 0.5;
vec2 dlProjCoordsExact = dlProjCoords / texelSize;
dlProjCoords = texelSize * ivec2(dlProjCoordsExact) + texelSize * .5;
vec2 dlProjSampleCoords = fract(dlProjCoordsExact);
DiscEdge leftEdge = findEdge(disc, dlProjCoords.xy, texelSize, vec2(-1, 0), dlMapLayer);
DiscEdge rightEdge = findEdge(disc, dlProjCoords.xy, texelSize, vec2(1, 0), dlMapLayer);
DiscEdge downEdge = findEdge(disc, dlProjCoords.xy, texelSize, vec2(0, -1), dlMapLayer);
DiscEdge upEdge = findEdge(disc, dlProjCoords.xy, texelSize, vec2(0, 1), dlMapLayer);
// todo: think what todo wiht both dir discontinuities?!!!
float x = interpolate(dlProjSampleCoords, leftEdge, rightEdge, disc.xy, vec2(1, 0), texelSize);
float y = interpolate(dlProjSampleCoords, downEdge, upEdge, disc.xy, vec2(0, 1), texelSize);
fragColor = vec4(x, y, 0, 1);
}

As you can see the function “bool hasDiscToDir(vec2 disc, vec2 dir)” looks so ugly.
Even as a 'findEdge"

The point is I want to add some improvements and unreadable hell make me hard think what I can do that.
Now I’m trying to find the structure that let me refactor shader for much friendly. Including format of data passes between stages, because of comparison of floats - the separate problem.
I’m thinking about bit masks or something like that.