ATI_fragment_shader parser anyone?

Does anybody know of a fragment shader language parser for the ATI_fragment_shader extension (i.e something like nVidia’s register combiner parser or directX’s ‘pixelshader’)?

Seems to be quite straight forward writing such a compiler (but I don’t have any intentions doing it, as I’m not working at ATI), why I’m a bit surprised I couldn’t find one on ATI’s website.

The no.1 reason why it doesn’t exist must be that there’s no need for one. The interface is so clean and obvious that a parsed interface wouldn’t have much of an advantage, IMHO of course

actually its very easy to parse through a shaderlanguage, you can code this for yourself (and its much more easy than the rc’s from nvidia, as the ati design is more simple imho)

go for it and code it. use std::string to make it easy for you… (i’ve written an xml parser in one evening… setting up some structured language for the atishaders and parse this, well… a weekend i guess (if everything works always ))

Hi

you could also use flex & bison for writing such a parser, its not so difficulat as it sounds, within a weekend I wrote a parser for GL_ARB_texture_env_combine and similar extensions, similar to nvparse.

Bye
ScottManDeath

I think there’s worth having an external fragment language so you don’t need to recompile your gl-core for every update to any shader. The language itself might be simple though.

Guess I’ll to do the work myself then (using lex/yacc). Luckily assembler parsers aren’t the most difficult ones.

thanks for the replies, guys!

and, yes, lex/yacc => flex/bison. i’ve never really understood the differences…

Hi

I think if you think further you can write a general “shading langauage” similar to quake 3 shaders where you can specifiy a shader that uses existing hardware as efficient as possible. So you can have a shader for nv_register_combiners, one for ati_fragment_shader, one for b_texture_env_combine and even one for opengl 1.0 without multitexturing but a quite bunch of render passes.

This could also expanded to support nv_vertex_programs and a ext_vertex shaders and perhaps dx vertex and pixel shaders.Later you could add support for OpenGL 2.0 shaders…

I think this would be quite a large task of work but the result would be an easy way to support multiple graphics accelerators with just writing a text file

Bye
ScottManDeath

I was under the impression than OpenGL 2.0 shaders will be supported with run-time assembly in anycase? (through the drivers). A common shading language?

Originally posted by Robbo:
[b]
I was under the impression than OpenGL 2.0 shaders will be supported with run-time assembly in anycase? (through the drivers). A common shading language?

[/b]

Hi

thats right but I think that a common
“shading language” is a superset because you could also declare textures with parameters(cubemap, filter), blending functions, depth test setup, stencil setup …
You could use for example when you want to render a textured,specular lit object without ext_separate_specular_color you can first render it with textured,diffuse only and then blend it with a specular, but untextured object.
If your driver doesn’t support GL2 you could implement a fallback mode with for example register combiners/texture shaders or fragment shaders.

Bye
ScottManDeath

A general shading language would ofcourse be state of the art, supporting old hardware and all… though such a project is a bit too heavy for a one man army. guess we’ll be waiting for gl 2.0 for that
It seems more realistic writing that ATI parser…

Also finding a language that combines ATI’s and nVidia’s fragment shading design is a very interesting task though…hrm.

I find it very problematic to get a shader compatible to older hardware. Because what do you do if someone specifies pixelshaders ?
Do you ignore them, do you try to simulate something similar?
This happens even with simpler texture combine operations. You can’t simulate every multitexture feature with multiple passes (if it is possible, i would like to know how).

So the only way seems to specify fallback code for simpler hardware.
And so you are back to write code for different Hardware.

So i will stay with a shader concept that fits to my game but isn’t very flexible, so can’t be used somewhere else.

This hopefully works on all kind of hardware and still looks as it meant to be.

Lars

Just to stray away from the subject a bit, people have been mentionning Lex and Yacc and Flex and Bison, but there is a more “modern” tool that exist which extremly nice called ANTLR.
http://www.antlr.org/

Antlr look kool enough, the grammar seems to be much more compact than common LR (I just briefly looked trough that website).

As we’re already away from subject …

An nice Lex/Yacc IDE can be found at http://www.bumblebeesoftware.com/. It’s much more convenient using than stand-alone lex/yacc.

Originally posted by Lars:
[b]
So the only way seems to specify fallback code for simpler hardware.
And so you are back to write code for different Hardware.

Lars[/b]

Hi

yes you are right, I think you can write a shader that shares commonalities as for example required textures and write fallback codes for different hardware. When you set the minimal requirements to for example multitexturing with ext_texture_env_combine I think you have a base where you can start of.When for example you would like to do dot3 bumpmapping without the texture_env_dot3 extension, you simply would skip it and use a low quality display.

My intention is to have a shading language that supports modern(needs to be discussed whats modern )graphics hardware with the possibility to fall back to (worst case) opengl 1.0 without extensions so that the app runs, but in a quite ugly way.
When you detect for example that you hardware supports 4 texture units you could could use single pass with multitexturing but on 2 textures hardware you would do multi pass and on opengl 1.0 you would just modulate lighting with the base texture.
You could write a shader implementation for nv_register_combiners and one for ati_fragment shaders.

I would not let the shading library the decision of which callback to use, instead i would like to to let the shadinf language determine which shader runs and then select the best(usually the first matching in list of possible implementations)

->ScottManDeath:

I agree in the solution of specifying different shaders for different hardware. It’s much more realistic and predictable than letting a library trying to do “as best a possible?” with a generic-all-feteaured language. I started working on the grammars for a shading setup language yesterday, and it doesn’t seem to be that awful a lot of work. Will require parsers for stencil-programs, blending-programs and ATI_fragment_shader though, but that’s pretty straightforward.

Hi

I could imagine a following sample shader (for specular lit textured objects

shader spec_tex
{
parameter texture base;
register_combiner reg_comb // this will be send to nvparse or done by the shader
{
!!RC1.0

rgb
{
  spare0=tex0*col0;
  spare1=col0;
}
out.rgb=spare0+spare1;

}
};

implementation // nv_register_combiners
{
{
bindtexture(0,base);
primary_color=lighting(ambient,diffuse);
secondary_color=lighting(specular);
fragment_op=register_combiner(reg_comb);
}
}

implementation // with ext_separate_specular
{
// first and only pass
{
bindtexture(0,base);
primary_color=lighting(ambient,diffuse);
secondary_color=lighting(specular);
fragment_op=GL_MODULATE;// here could be also register_combiner …
}
}

implementation // gl 1.0
{
{// diffuse + ambient pass
bindtexture(0,base);
primary_color=lighting(ambient,diffuse);
fragment_op=modulate;
}
{// specular pass
// no textures specified, no texturing
depth_func=GL_EQUAL;
blend_func=GL_ONE,GL_ONE;
primary_color=lighting(specualar);
}
}
}

To us this in the app I would like to do following

class Texture tex;
tex.Load(“grass.tga”);
class GeoObject sphere; // an object that rendere geometry, in simple case it could be a glutSpherm but later there would be a vertex array

ShaderLib::LoadShader(filename);
ShaderLib::SelectShader(“spec_tex”);
ShaderLib::SetParameter(“base_tex”,tex);
ShaderLib::Render(sphere);

this are just thoughts, I think there will be the need of discussion what the language should be possible to do, how the style should be, wich tokens are necessary

I did not look into the ATI spec of ext_vertex shader, but perhaps it would be possible to write a parser that parses a NV_vertex_program?

Bye
ScottManDeath

[This message has been edited by ScottManDeath (edited 05-30-2002).]

[This message has been edited by ScottManDeath (edited 05-30-2002).]

Shader myFunkyShader
{
Shader(NV15) //i.e Geforce2
{
VertexStream
{
v[0] = glVector3; //position (well, as long as 3 componenets are availible, this might be anything)
v[1] = glVector3; //normal
v[2] = glVector2; //uv

		//note that this shader can be used with any stream that produces at least these three components
	}

	TextureStream
	{
		t[0] = glTexture2D;
		t[1] = glTextureCubeMap; //only two texture units on NV15..
	}

	Pass(0)
	{
		VertexStateProgram
		{
			Constants
			{
			}
			Source = "D:\\shaders\\myFunkyShaderNV15.vsp"; //nvparse
		}

		VertexProgram
		{
			Constants
			{
				c[0] = glModelviewProjectionMatrix; //the renderer will use glTrackMatrixNV()
				c[4] = glModelviewMatrix;			//and here too.
				c[12] = glInverseModelViewMatrix;	//and here..
				c[13] = glSceneAmbient;				//custom information, tracked internally by the renderer
				c[14] = glMaterialDiffuse;			//custom tracking..
				c[15] = glMaterialEmissive;			//and custom tracking..
				c[17] = glPointLight0;				//custom light source tracking (always in eye space)
				c[16] = [0.5, 2.0, 1.0, 9.854];
			}
			Source = "D:\\shaders\\myFunkyShaderNV15.vp"; //nvparse
		}

		FragmentProgram
		{
			Constants
			{
				c[0] = glFogColor;
			}
			Source = "D:\\shaders\\fragment\\myFunkyShaderNV15.rc"; //nvparse
		}

		BlendProgram
		{
			Constants
			{
				c[0] = [1.0f, 0.5f, 1.0f, 1.0f];
			}
			Source = "D:\\shaders\\myDefaultBlend.bp"; //own parser
		}
		StencilProgram
		{
			Source = "D:\\shaders\\myStencilCode.sp"; //own parser
		}
	}//Pass(0)

	Pass(1, 2, N,..)
	{
	}

}

Unresolved:
There’s not yet availible to select specialized vertex programs for different lighting conditions
(1 dir-light, 13 spot lights, 8 point lights). Dunno how to solve this atm…

//this would be in another file ofcourse. just pseudo code, any decent script language could handle this parsing…

Object myStaticBuilding
{
VertexStream = lightwaveIO.read(“D:\objects\house1.lwo”);
TextureStream0 = imageIO.read(““D: extures\house1.dds”);
TextureStream1 = imageIO.read(”“D: extures\cubemap0.dds”);
Shader = shaderfactory.createShader(“D:\shaders\myShaderY2k.shd”);
}

//or something like this could work…

Object myParticleSystem
{
VertexStream = effects.ParticleFountain(param, param, param…param_n); registered vertexstream called ParticleFountain
//the particlefountain vertexstream is registered in a dll or whatever, just requires a certain interface to be
//registered… (probably by sub-classing the internal VertexStream interface)

Shader = "D:\internal_shaders\particlesystemfountain.shd";
TextureStream
{
	t[0] = effects.TextureNoise(param, param..); //subclassing the Texture2D interface for example..
	t[1] = "d:	extures\particle1.tga";
}

}

This example wouldn’t do anything useful anyhow, I just wrote it down as a mindmap…

Hm… It might look at bit confusing why I didn’t specify the vertex-programs and fragment-programs within the shader, but there’s a good reason not too. Having them separated I can use the same vertex-program in different shaders (maybe not -very- likely, but…), the same goes for the fragment-programs, blend-programs, stencil-programs etc… Once compiled they’ll be identified with an Id (which would be a compiled display list Id or a VP id or a VSP id).

These notes are about one day old, so please be gentle with the idea…

Hi

I thought also about separating register_combiners and vertex_programs but I thought it would be not very much used but perhaps it could be done in a way as C++ scoping

example

shader0.txt:
shader spec_tex
{
register_combiner reg_comb
{

}
implementation
{
{// first pass
fragment_op=register_combiner(reg_comb)
}
}

}
}

shader1.text
shader shader_two
{
extern register_combiner extern_rc = spec_tex::reg_comb;
implementation
{
{
fragment_op=extern_rc
}
}
}

to make this work I think there is the need of managing shaders independent from the parser. I would like to develop the shader classes, being able to setup them with c++ code. If that works, there could be the parser that setup shaders. I can imagine a kind of intermediate ‘byte’ code that is accepted by a shader to setup itself and the parser generates the byte code for the shader library. An advantage would be that you can test your shader as a text file and than later you decide to save it as byte code to reduce storage amount and prevent the unauthorized editing.

I think it would be useful to develop the shaderlib as a DLL WITH dynamic loading, so you can exchange the dll and the shader files when for example a new graphics card is supported by the shader lib.

I thought also about the possibility to setup vertex programs constant by the shader lib to track lighting/material properties.

supose you have a particel system where a particle should fade by reducing its alpha value, but the alpha value should be controlled by the particle not by the shader.
I would like to have a kind of parameter tracking from the app to the shader lib, as similar as your vertex shader constants

shader particle
{
parameter // parameters that will be accepted later in the shader
{
texture part_tex;
vec4f part_color;
}
implementation
{
{
blend_func=GL_src_alpha,GL_one_minus_src_alpha;
bind_texture(0,part_tex);
fragment_op=modulate;
primary_color=part_color;
}
}

}

you app code would be

particle::render()
{
ShaderLib::SelectShader(“particle”);
ShaderLib::SetParameter(“part_color”,vec4f(1,1,1,1-(cur_time/life_time));
ShaderLib::SetParameter(“part_tex”,mytexture);
ShaderLib::SetGeometry(particle_geometry); // could be a vertex buffer …
ShaderLib::Render();
}

Bye
ScottManDeath

Guys, have a look at http://graphics.stanford.edu/projects/shading

Michael