Apologies for the long post. Two issues I’d like to raise:
- mandatory detailed conformance reporting
- community-assessed OpenGL implementation performance/quality
I’d like to see Khronos require all implementors of the OpenGL spec release to the public the full conformance test results for every driver release. Here’s why…
In a thread in “OpenGL.org » Forums » DEVELOPERS » OpenGL coding: advanced » Memory usage”, the issue of texture proxies came up:
I wouldn’t rely on proxies… We had some reports suggesting that IHV don’t take them too seriously.
The poster is clearly trying to do the right thing. Unfortunately, without considerable effort, there’s no way for a reader to verify or refute the claim (which vendors, how badly broken, how recently, etc)
So texture proxies are required by the spec, but because of a report that one or more implementors decided not to support it in a useful way, developers may stop using part of standard OpenGL functionality.
This isn’t just about proxy textures. There are many places the spec leaves behaviour unspecified. Similar doubts may arise elsewhere. If the standard is undermined, everyone loses.
I couldn’t find any information about if or how well texture proxies are tested in the OpenGL conformance suite. I couldn’t find any test results either. If the average reader didn’t have time to develop their own test and run it on a variety of cards, they’d understandably accept the wisdom of the poster, rated as “OpenGL Pro” by the forum software.
If Khronos required that the full OpenGL conformance test results be made public for every driver version (or other implementation) OpenGL implementors released, situations like this might be avoided.
I suspect any competent OpenGL implementor runs the conformance tests prior to a driver release in any case, so not much extra work would be required.
If test results were collated in one place by Khronos (c.f. SPEC and its results), public scrutiny of the results by developers and consumers might “encourage” less “meticulous” implementations in the “right direction”.
If you believe in free markets.
But conformance to the specification is sometimes not enough…
Perhaps the conformance tests wouldn’t have revealed that in some implementations, texture proxies conform to the API but provide little useful information, create performance issues, etc.
The OpenGL spec is deliberately non-specific in some places, to allow implementors to adopt approaches that match their priorities, architecture, etc. It’s not clear that adding a minimum functional requirement for texture proxies would have been possible or would have helped in the example above.
Developers (and ultimately the consumers they serve) often need more than just mandatory conformance information. Other issues like coverage of optional features (e.g. which texture formats) and performance/quality of implementation (e.g. texture proxies) are important.
Perhaps developers frustrated by continually having to write feature and performance tests of their own could pool their code to create a community-based conformance and performance tests available to all. Add a website for the results, forums for discussion (e.g. right of reply by OpenGL implementors, clarification by specification authors, etc) and you’d have a really useful resource.
This should be applied to extensions too. The extension registry doesn’t give any indication of how widely, how recently or how effectively an extension is supported.