You are right and you are not being helpful though. Rather than just point out what is wrong, why not give suggestions to fix an issue.
I am being helpful; I’m pointing out the flaws in your proposal. It’s your responsibility to find fixes for them because it’s your idea.
Remember: I’m perfectly happy with shading_language_include (though I wouldn’t be averse to an object wrapper for filesystems). I think that your proposal is over-designed and needlessly complex for the needs of the vast majority of users, for the reasons I have pointed out. I could have been actually unhelpful by simply not commenting on this proposal at all. In which case you would not have known about the flaws in it and never have designed fixes for them.
Stop making this personal. I am not attacking you; I’m attacking your idea. Respond to what is written and not the person doing the writing.
Simple and easy to do. Also honest saying up front what is going on.
Except that it is neither simple or easy for the user. It may say “up front” where included strings are coming from for a particular compile, but the user now must implement 4 callbacks. If new functionality comes down the pipe for shaders that uses a new compilation routine (see separate_shader_objects), that new functionality will have to provide new entrypoints in order to explicitly provide inclusion. And so on.
Every time something is corrected in this proposal, it becomes more and more complex for the user to implement. It went from 3 callbacks to four. From 1 user-defined object to 3. Inclusion really should not require this much effort on the part of the user.
merging of different strings to common paths/filenames is undefined beyond handling “…” and “.”. This is most troubling since different environments have different rules.
I don’t know what more definition you want. If you’re talking about case sensitivity, that’s effectively implicit in the spec: string compares are case sensitive. It would be good to have this stated explicitly, of course. But otherwise, there’s nothing more to specify.
This will not result in problems when trying to mimic a filesystem. If you’re mimicing a Windows filesystem, which is case insensitive, then nothing changes. Files that had different names under case insensitivity will continue to have different names under case sensitivity. And if you’re mimicing a case sensitive filesystem, again, everything is fine because shading_language_include is case sensitive.
And if you’re trying to mimic a filesystem that uses different path separators or hardlinks or some-such… well, tough. That’s not something the vast majority of people need to do.
This extension was designed to provide the greatest utility while having the least implementation and user burden. And it strikes a very good balance. No, it doesn’t do everything you could possibly want to do with a filesystem; but it does do enough.
I can definitely see where some framework create their own “library” of headers and they wish to use. That requires that they make sure their paths are unique, essentially what I am talking about is “path” namespace pollution.
Really, how much inclusion are you expecting to exist? I don’t have these kinds of include issues in my C++ projects, let alone for the relatively small numbers of shaders that people will be using. Reasonable conventions ensure that this won’t happen for any real-world applications. The scope of shaders simply doesn’t merit it.
Not only that, you can always test whether a path already exists in the filesystem. Yes, that means a library will throw an error when it attempts to be used with a conflicting one. But this occurrence will be so rare anyway that doing anything more than just failing would not be worthwhile.
Is it possible for conflicts and so forth to happen? Absolutely. But is this eventuality likely enough to be worth the effort that a user would have to spend to write a virtual file system? I highly doubt it.
Also, let’s consider how your proposal handles this situation.
Let’s say you have two layered libraries that provide their own includes. And you have some shaders that need to use includes from either library as well as some private includes of your own. How do you gain access to their library of includes? They would have to provide access to these includes. But since there is no agreed-upon protocol for this, each library can implement this in their own way. One library might just have some loose files, while another may have put all their include shaders in an XML file format.
So your filesystem code now needs to parse whatever their mechanism for storing these things is. A “simple” stdio implementation of a filesystem goes right out the window; you must use some form of virtual file system. That means writing a lot of code. In the case of name conflicts, you have to decide which library gets primacy.
Isn’t that a rather large quantity of work for the user? It certainly isn’t “simple”, which is how you initially described this proposal. If global state and a few reasonable conventions can solve the problem without excess effort on any party, why not use that?
Now there is another way besides from what I have suggested to handle the global state issue: introduce an object “virtual GL filesystem”:
That still requires direct interference with the creation and life-span of a shader object. So it can’t work with separate_shader_objects. Nor can it work with any extension that creates programs internally based on shader source that you provide unless it also exposes this interface.
Note that my suggestion above already has this functionality in it.
No, it does not. It simply forces the user to play their own variant of “the horrific #ifdef game” in his filesystem implementation instead. It doesn’t solve the problem; it just hands it to someone else.
Adding a new entry points is NOT a big deal, GL_EXT_direct_state_access adds a ton of them, and it did not matter. The .dll/.so made was not that much bigger.
It’s not a question of the number of entrypoints (though DSA defined quite a few functions of questionable merit. Seriously, was there a need for functions like glMultiTexParameteriEXT, which still depends on bound state even though getting rid of that was the point of DSA?). It’s a question of ease of use and integration with existing code.
You spoke of layered libraries. The global state of shading_language_include makes it easier to use layered libraries with includes precisely because it doesn’t add new entrypoints for compiling. Yes, it also means that layered libraries could in fact interfere with your filesystem. But the benefits of ease-of-use outweigh the risks.
Extensions need to work well with existing functionality and code. We’ve seen this time and again; people will not radically rewrite their rendering code on a whim. By making shading_language_include work with existing functions, it means that users don’t have to rewrite their code. They just add some simple initialization code and everything works perfectly. Because of this, they are more likely to actually use the feature.
Yes, this mode of thought lead directly to the weirdness of how array buffers are attached to attributes, the bind+gl*Pointer paradigm that confuses new users constantly. Even so, the ability to switch back and forth between buffer objects and client arrays with minimal code changes was instrumental to the widespread adoption of buffer objects. Yes, by now they should have corrected the API oversight, but at the time, it was a perfectly defensible and reasonable action.
And, in this case, there is no API strain caused by shading_language_include. The API is not made more obscure or difficult to use. It is plainly evident exactly what is happening: the filesystem that the user specified to OpenGL is being used for shader string inclusion, for all shader compilation. While the specific effect of shader compilation is now bound to certain global state, that’s essentially what you want when you’re talking about an include mechanism. Ultimately, any form of inclusion means that the result of compilation will be affected by state external to the shader object itself. Whether it is global GL state or some user-defined code.
Global state is not always a bad thing. Particularly when you’re dealing with a concept that is fundamentally global.