The Khronos Group today announced the ratification and public release of the Khronos Data Format Specification 1.0. This new standard provides precise mechanisms to generate machine-readable format descriptions of repetitive data, such as pixels, enabling standards and software to cleanly communicate and interoperate. This royalty-free specification will be used across multiple Khronos API initiatives, and is open to be used by third party standards and applications. The Khronos Data Format Specification and additional information is available at: https://www.khronos.org/dataformat.
“The minefield of data formats and their interpretation by each API is one of the most challenging areas of integration of device components, and interoperation between them,” said Alon Or-bach, chair of the EGL Working Group. “The Khronos Data Format Specification provides a clean and complete descriptor to describe all common formats and enables APIs to define clear mappings to them. This will improve life for device integrators, and be an invaluable tool for applications that depend on data flowing between different hardware blocks, such as computer vision and augmented reality.”
Today, many standards use proprietary enumerated data descriptions that are not extensible, and are often incompletely specified, resulting in data conversion errors and incompatibilities. By using a descriptive data format, software can flexibly and reliably use a large number of possible formats without specifically coding for each one. The Khronos Data Format Specification supports versioning and extensions, as well as storing common data interpretation details to avoid the need for separate metadata.
This specification has already been used to describe precise descriptions for Vulkan™ and OpenVX™ internal data formats. APIs can also define extensions with data format descriptors to be used for both interoperation between multiple standards and for describing user-visible data.
So, I have just read through the dataformat.1.0 PDF. I am not going to pretend that this first look is enough to catch anything that might be wrong or missing, but I was specifically looking for a particular use case, and not having found a definitive answer or example, I thought I would bring it up.
One type of common data in the graphics field, at least in the 3D renderer world, is ptex data. My specific question is how would we handle the ptex use case when the ptx’s model is a triangular mesh. In ptex, each polygon gets its own texture, and in the case of triangular meshes, these textures are trangle shaped, meaning each line has a different size.
Would this be an example of a use case for the undefined namespaces, and only worry about the total size of the texel block? Or is there some other method I am missing that is painfully obvious to others?
Thanks for all the hard work, and this does look like it will cover a huge number of use cases, so cheers.
Hi Bill. Thank you for taking an interest in this spec.
So, the way to think about the Khronos Data Format Spec is that it would have been called the “Pixel Format Spec” if we hadn’t also wanted to use it to encode things that live in linear buffers or in voxel grids, and if it wasn’t also useful for storing normals, vectors, etc. “Data” was the fallback term. The important distinction from the perspective of ptex (which is a good example) is that this specification is only about how to interpret the texel data, not where that texel data may be stored in memory.
Thought of as a “texture format”, it has to be that way - two textures can be the same “format” (by most conventional interpretations) if they’re a different size, or even a different dimensionality. For internal representations many types of graphics hardware have their own proprietary tiling formats, which means that the simple “x + stride * y” formula is a vast oversimplification. This specification is about a smaller problem: “I’ve found my texel (however I managed to read it), but I don’t know how to interpret it”. This isn’t quite such a cop-out as it sounds, because most applications would end up storing image-specific dimensions and stride even if they’re using the simple case - and trying to incorporate that information into the “format” mostly reduces flexibility. For a triangular ptex image, the line-dependent stride would be handled separately from the “format”, and as part of the coordinate-to-memory-location mapping. (I can imagine something complicated if such a triangular image were compressed with a rectangular block texture format, but I think the handling of that would be pretty implementation-specific).
I can find a description of the ptex file format, but here we’re talking about in-memory representations (which mostly rules out variable-rate compression if you want fast look-ups); I’d heard of ptex, but I’ve not used it. If I’m interpreting this correctly, ptex lets you define a texture containing an arbitrary number of channels, (up to) one of which is identified as alpha, and all of which share a datatype of uint8, uint16, float16 or float32. The numbers stored in those channels are, as far as I can tell, implementation-specific in ptex (it’s up to your shader what you want to do with them). Where the Khronos Data Format Specification comes in is to define what those channels mean. Were they red, green, blue? If so, what order? And what red, green and blue? When you encoded an 8-bit 0…255 value, did you actually mean “255” to mean 255, or to mean 1.0? Were you actually encoding HSV? Was the texture gamma corrected (what transfer function does it have)? It seems that ptex defines that the RGB values should not be premultiplied by alpha on the first level, but should be on later levels; that’s actually a complication, which would mean that you’d need a different data format descriptor for the first level than you would for the rest (but you would also need different GPU or host function behaviour for these cases).
If you generated your ptex file and you’re consuming it, this doesn’t matter so much: you know how to interpret the data you put in there. If you want someone else’s library to handle it properly, though, some means of describing what’s actually stored in the ptex can be useful.
My expectation would be that you could create a data format descriptor that has a “sample” for each channel. If, as suggested in the file format, you have each channel stored in a separate plane, you’d refer to the start of each plane somewhere in whatever was looking up the ptex values for you, and each channel would correspond to plane in the data format descriptor - this means that the basic data format descriptor can only store ptex files up to 8 planes (this was a trade-off for the size of a data format descriptor for common formats), but you can, of course, use an extension to do what you want beyond that. Each plane would then contain 1, 2 or 4 bytes. If the channels are stored in “chunky” format (consecutively within a pixel, rather than planar) there is no such restriction. Then you’d need one sample for each channel, one of which may be labelled as alpha, and the rest as needed.
I hope that helps (and that I understood the request!)
Thank you; I hope you (and others) find it useful. And I hope I never again end up banging my head against the table when reverse-engineering what someone meant when their documentation just said “YUV”, obviously. If there are cases that are a problem to represent in the initial specification, or if there is anything that needs clarification, please raise them here, and we’ll try to incorporate them in the next revision.