Note: this is not a “real” extension specification. I wrote this in my spare time. I’m simply dumping it off here for comment; I make no claim that this is a full and complete extension spec. And maybe the ARB will see it and adopt it sometime before the heat death of the universe…
GL_EXT_Program_Data_Object
Status
Incomplete
Version
Last Modified Date: 8/14/2008
Author revision: 1
Number
N/A
Dependencies
This extension is written against the OpenGL 3.0 specification. It is written so as to avoid providing entrypoints that would be deprecated for similar reasons as other deprecated GL 3.0 features. It will also not extend said entrypoints.
This extension trivially interacts with ARB_geometry_shader4.
Overview
One of the pain points involved in using shader programs in OpenGL is that each fully linked program encompasses not only the compiled and linked program executable, but the block of uniform and texture state for that program. This tightly couples two concepts that should be distinct from one another: the executable code and the data that acts on it.
Additionally, the fully linked program must contain all programmable stages. It is often the case that the user has a vertex program that could be compatible with a number of fragment programs. This mixing-and-matching can be done in OpenGL, but it creates a huge number of fully linked programs. There is no light-weight mechanism for linking two programs together.
This extension attempts to solve both of these problems.
It does this by introducing an additional object, called a Program Data object. A Program Data object is built from Shader objects much like a Program object. That is, a Program Data object is a fully linked part of the programmable pipeline. It must provide at least one full stage of the pipeline, and it cannot consist of any partial stages. However, unlike Program objects, Program Data objects do not have to provide all stages of the pipeline. They also do not provide uniform state, attribute state, or any of the other state data.
Program objects can be linked from Shader objects as normal, or they can be created from one or more Program Data objects. Program objects cannot be created from both; attempting to link a Program object that has both Shader objects and Program Data objects bound to it is an error.
A Program object linked from Program Data objects is created exactly as though the component Shader objects of those Program Data objects were used. However, linking a Program with Program Data objects is a lightweight operation, compared with linking a Program from Shader objects. The code generation has been done in the Program Data linking stage; the only checks that need to be done are for the connectivity of uniforms, and stage input/output variable linking.
Program objects retain all of their state data as normal.
This extension also deprecates the mechanism of linking Program objects directly from Shader objects.
New Procedures and Functions
uint CreateProgramData( void );
void DeleteProgramData( uint programData );
boolean IsProgram( uint programData );
void AttachShaderProgramData( uint programData, uint shader );
void DetachShaderProgramData( uint programData, uint shader );
void LinkProgramData( uint programData );
void AttachProgramData( uint program, uint programData );
void DetachProgramData( uint program, uint programData );
void GetProgramDataiv(uint programData, enum pname, int *params);
void GetProgramDataAttachedSahders(uint programData, sizei maxCount, sizei *count, uint *shaders);
void GetProgramDataInfoLog(uint programData, sizei bufSize, sizei *length, char *infoLog);
Additions to Chapter 2 of the OpenGL 3.0 Specification (OpenGL Operation)
Modify subsection 2.20, Vertex Shaders
(modify paragraph 1, p88): …from all the compiled shader objects attached to the program. Shader objects can also be attached to a program data object, which can be linked like a program object. Program object executables can be created from one or more linked program data objects.
(modify paragraph 2, p88): …A single program or program data object can contain both vertex and fragment shaders.
Modify subsection 2.20.1, Shader Objects
Insert subsection 2.20.1a between 2.20.1 and 2.20.2
2.20.1a Program Data Objects
The shader objects that are to be used by the programmable stages of the GL can be collected together to form a program data object. A program data object represents the compiled and linked code for one or more programmable stages.
The name space for program data objects is the unsigned integers, with zero reserved for the GL. This name space is shared with progrma objects and shader objects.
To create a program data object, use the command
uint CreateProgramData( void );
The program data object is empty when created. A non-zero name that can be used to reference the program data object is returned. If an error occurs, zero will be returned.
The command
void AttachShaderProgramData( uint programData, uint shader );
attaches a shader to the program. The error INVALID_OPERATION is generated if shader is already attached to programData.
Shader objects may be attached to program data objects before source code has been loaded into the shader object, or before teh shader object has been compiled. Multiple shader objects of the same type may be attached to a single program data object, and a single shader object may be attached to more than one program data or program object.
To detach a shader object from a program data object, use the command
void DetachShaderProgramData(uint programData, uint shader);
The error INVALID_OPERATION is generated if shader is not attached to programData. If shader has been flagged for deletion and is not attached to any other program object, it is deleted.
In order to use this program data object to link program objects, the program data object must be linked. The command
void LinkProgramData( uint programData );
will link the program data object named programData. Each program data object has a boolean status, LINK_STATUS, that is modified as a result of the linking. This status can be queried with GetProgramDataiv (see section 6.1.15). This status will be set to TRUE if a valid executable fragment is created and FALSE otherwise. Linking a program data object can fail for a variety of reasons as specified in the OpenGL Shading Language Specification. Linking will also fail if one or more shader objects attached to programData are not compiled successfully, or if only a part of a programmable stage is specified by the attached shader. If LinkProgramData failed, any information about a previous link of that program data object is lost. Thus, a failed link does not restore the old state of programData.
Each program data object has an information log that is overwritten as a result of a link operation. This information log can be queried with GetProgramDataInfoLog to obtain more information about the link operation or the validation information (see section 6.1.15).
After a successful link of a program data object, applications are free to modify attached shader objects, compile attached shader objects, attach additional shader objects, and detach shader objects. These operations do not affect the link status or executable code of the program data object.
Program data objects can be deleted with the command
void DeleteProgramData( uint programData );
If programData is not attached to any program object, it is deleted immediately. Otherwise, programData is flagged for deletion and will be deleted when it is no longer attached to any program object. If an object is flagged for deletion, its boolean status bit DELETE_STATUS is set to true. The value of DELETE_STATUS can be queried with GetProgramDataiv. DeleteProgramData will silently ignore the value 0.
Modify subsection 2.20.2, Program Objects
(modify the last paragraph, p89): The shader objects or program data objects that are to be…
(modify paragraph 1, p90): … if shader is already attached to program. The error INVALID_OPERATION is generated if program has any program data objects attached to it.
(insert after paragraph 3, p90): To attach a program data object to a program object, use the command
void AttachProgramData( uint program, uint programData );
The error INVALID_OPERATION is generated if programData is already attached to program. The error INVALID_OPERATION is generated if program has any shader objects attached to it.
Program data objects may be attached to program objects before shader objects have been attached to them, or before they have been successfully linked into an executable fragment. Multiple program data objects may be attached to the same program object, and a single program data object may be attached to more than one program object.
To detach a program data object from a program object, use the command
void DetachProgramData( uint program, uint programData );
The error INVALID_OPERATION is generated if programData is not attached to program. If programData has been flagged for deletion and is not attached to any other program object, it is deleted.
(modify the last paragraph, p90): In order to use the shader objects or program data objects contained in a program object…Linking will also fail if one or more of the shader objects, attached to program are not compiled successfully, if one or more of the programData objects, attached to program are not linked successfully, if two or more programData objects specify the same programmable stage, or if more active uniform…
(modify paragraph 3, p91): While a program object is in use, applications are free to modify attached shader objects, program data objects, compile attached shader objects, link attached program data objects, attach additional shader or program data objects, and detach shader and program data objects. These operations do not affect the link status or executable code of the program object.
Additions to Chapter 6 of the OpenGL 3.0 Specification (State and State Requests)
Modify subsection 6.1.15, Shader and Program Queries
(modify the title): Section, Program Data, and Program Queries
(modify the first paragraph after the title, p333): State stored in shader, program data, or program objects can be queried by commands that accept shader, program data, or program object names. These commands will generate the error INVALID_VALUE if the provided name is not the name of either a shader, program data, or program object, and INVALID_OPERATION if the provided name identifies an object of the other type…
(insert after paragraph 2, p334): The command
boolean IsProgramData(uint programData);
returns TRUE if programData is the name of a program data object. If programData is zero, or a non-zero value that is not the name of a program data object, IsProgramData returns FALSE. No error is generated if programData is not a valid program data object name.
The command
void GetProgramDataiv(uint programData, enum pname, int *params);
returns properties of the program data object named programData in params. The parameter value to return is specified by pname.
If pname is DELETE_STATUS, TRUE is returned if the program data has been flagged for deletion and FALSE is returned otherwise. If pname is LINK_STATUS, TRUE is returned if the program data was last compiled succesfully, and FALSE is returned otherwise. If pname is INFO_LOG_LENGTH, the length of the info log, including a null terminator, is returned. If there is no info log, 0 is returned. If pname is ATTACHED_SHADERS, the number of objects attached is returned.
The command
void GetProgramDataAttachedSahders(uint programData, sizei maxCount, sizei *count, uint *shaders);
returns the names of the shader objects attached to programData in shaders. The actual number of shader names written into shaders is returned in count. If no shaders are attached, count is set to zero. If count is NULL then it is ignored. The maximum number of shader names that may be written into shaders is specified by maxCount. The number of objects attached to programData can be queried by calling GetProgramiv with ATTACHED_SHADERS.
(modify the last paragraph, p335): A string that contains information about the last compilation attempt on a shader object, last link attempt on a program data object, or last link…can be obtained with the commands
void GetShaderInfoLog( uint shader, sizei bufSize, sizei *length, char *infoLog );
void GetProgramDataInfoLog(uint programData, sizei bufSize, sizei *length, char *infoLog);
void GetProgramInfoLog( uint program, sizei bufSize, sizei *length, char *infoLog );
These commands return…The number of characters in the info log can be queried with GetShaderiv, GetProgramDataiv, or GetProgramiv with INFO_LOG_LENGTH. If shader is a shader object, the returned info log will either be an empty string or will contain information about the last compilation attempt for that object. If programData is a program data object, the returned info log will either be an empty string or it will contain information about the last link attempt. If program is a program object…
Additions to Chapter 1 of The OpenGL Shading Language, version 1.30 (Introduction)
Modify section 1.3, Overview
(modify the second paragraph of the section): Independent compilation units written in this language are called shaders. A program is a complete set of shaders that are compiled and linked together. Program data is a set of shaders that fully specifies one or more programmable stages. The aim of this…
Additions to Chapter 3 of The OpenGL Shading Language, version 1.30 (Basics)
Modify section 3.2, Source Strings
(modify the first paragraph of the section, p8): …when it concatenates the strings to form a single shader. Multiple shaders can be linked together to form a single program or program data.
(modify paragraph 4, p11): …as targeting version 1.10. Different shaders (compilation units) that are linked together in the same program or program data must be the same version.
Additions to Chapter 4 of The OpenGL Shading Language, version 1.30 (Variables and Types)
Modify section 4.2, Scoping
(modify paragraph 1, p27): …of the same language (vertex or fragment) that are linked together to make a single program or program data.
Modify subsection 4.3.4, Input
(modify paragraph 1, p 30): …There is an implementation dependent limit on the number of locations that can be used, and if this is exceeded it will cause a link error duing program link, but not during program data linking. (Declared input…
Modify subsection 4.3.5, Uniform
(modify paragraph 6, p30): …must match across all shaders that are linked into a single executable or executable fragment.
Additions to Chapter 6 of The OpenGL Shading Language, version 1.30 (Statements and Structure)
Modify section 6.1, Function Definitions
(modify paragraph 5, p53): …but one shader in a set of shaders linked together to for a single shader executable or executable fragment must…
Interactions with ARB_geometry_shader4
Geometry shader4 simply provides an additional programming stage. Program data objects can work with these as normal.
Issues
1: What should this extension be called?
There are several possibiliites. It could be called “Program_Stage_Separate”, since part of the purpose of this extension is to separate the stages of a Program. It could be called “Program_State_Separate”, since it provides separation between a program’s data and its uniform state.
However, there is a long-standing precident for extensions revolving around a new object type to use the name of that object in its name. That is why we settled on “Program_Data_Object”
2: This extension attempts to allieviate 2 API issues simultaneously. It may be more reasonable to split this into multiple extensions, each focused on a specific problem domain.
Discussion: This solution elegantly solves both problems.
3: Should it be legal to link one or more Program Data objects to one or more Shader objects to form a coherant program?
Discission: No.
Removing this makes the specification that much simpler and straightforward. It also makes no sense to do so; the primary purpose of Program Data objects is to be able to link Program objects more efficiently. If one is using Shader objects, this will require the usual laborious process of Program object linking. So combining the two is a semantically meaningless operation, even if it were defined.
4: Why add an object between Shaders and Programs? Is there not some other way to solve this problem?
Rationale: The main reasons to do it this way are as follows. It creates the least API burden; the only functions made redundant are those that bind Shader objects to Program objects. The only new entripoints are those for the new Program Data object. The Program object already has all of the state accessor functions that would be needed to set uniforms, querry attributes, and the like. Adding a new object that has all of these would expand the number of API functions needlessly.
I also think it would be easiest to implement this way. It leverages the existing Program and Shader infrastructure, adding an object and APIs only where there is a deficiency. Initial implementation
5: Should Program Data objects be limited to a single stage, or should they allow multiple (complete) program stages?
Discussion: Logically, putting multiple stages in a Program Data object is possible. Additionally, since only one of the purposes of this extension is to separate stages, it probably should be on the user as to which stages they wish to separate and which ones they wish to have bound together. The separation of program state from the program code is something a user might want if they use a lot of unique stage-to-stage programs.
If it is a burden to implementations, we can restrict a Program Data object to a single stage. It is not that significant an issue for a user to have to separate their stages.