Write your own Einstein@home screensaver

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6542
Credit: 287066698
RAC: 97422

Basic Shaders for

Basic Shaders for Dummies

It looks like I'll be able to get away with bland & basic shaders and pipeline configuration. In fact I will/am abstracting all the key GLSL constructs in a C++/OOP framework. I reckon I'll go up to a 'PipelineConfig' class which will comprise all shaders and associated data structures/buffers for a given display task eg. render the Earth, render the constellations etc. The key point is to regularise all the annoying GL/GLSL interfacing specific workload within and allow a higher level of thinking.

[ For that matter I think the framework can/will generalise well beyond our E@H screensaver focus as applied here ie. a multi-target cross-compile build system for OpenGL ES. What are the possible applications of that ? :-) :-) ]

GLSL has a superficial appearance to C/C++/Java, but don't be fooled! I think it has that look & feel to make us comfortable, but the paradigm is truly different.

- identifiers/symbols are case-sensitive, composed of one or more alphanumeric characters plus the underscore ( '_' ). The first character must not be a numeric digit nor the underscore. Consecutive underscores ( '__' ) are disallowed anywhere within a symbol, as these are GLSL reserved. As are those beginning 'gl_'.

- whitespace is free-form, thus language parsing is indifferent to whether the symbols are separated by a space, multiple spaces, tab character, CR and/or LF, combinations thereof. But be smart and use the given freedom to make it easy to understand at a glance.

- shader code is merely a single ASCIIZ terminated string or an array of same ( eg. one string per 'line' ). To be exact a single string is an array of strings with only one/first element.

- statements end with a semicolon.

- flow-control/iteration/test is per C except no 'goto' ie. functions with expected semantics, if-else, switch-case​, for​, while​, do-while, break​, continue​ and return​.

- recursion is illegal.

- comments are :

[pre]// Single line like this, or

/*
Multi-line beasts.
For as many as you like,
exhaustion limited.
*/[/pre]
- the first non-whitespace line ought be like so :

[pre]#version 330 core[/pre]
.... declaring the GLSL version and profile. If you don't you'll get 110/1.1 core, probably not what you wanted. At GLSL compilation time - which is at run time for your executable - one will be held to that declaration. There is a correspondence b/w OpenGL and GLSL versions :

[pre]GL GLSL
2.0 1.10
2.1 1.20
3.0 1.30
3.1 1.40
3.2 1.50[/pre]
where for OpenGL 3.3 and above, the corresponding GLSL version matches the OpenGL version eg. GL 4.1 uses GLSL 4.10, which appears in the shader as 410.

- the executable part is within this construct :

[pre]void main() {
// Doing stuff here.
}[/pre]
... where 'void main()' is just syntactic sugar to resemble C/C++, as there is never a return type, no parameters are ever passed as an argument list, and the name is 'main' for all shaders. So it's merely a delimiter of the code statements with familiar appearance.

- as opposed to data declarations. You can have both before the code block or within. For those declared prior to the 'main' block eg.

[pre]#version 330 core

in vec4 vPosition;
in vec4 vColor;

out vec4 color;

uniform mat4 ModelViewProjectionMatrix;

void main() {
color = vColor;
gl_Position = ModelViewProjectionMatrix * vPosition;
}[/pre]
these correlate to values being fed by the required Vertex Array Object - the 'in' variables - or set/updated on occasion by the application - the 'uniform' variables. The 'out' variable is passed to ( or is an alias for ) a variable of the same name which is an 'in' further along. Here we are evidently simply passing a color value through a vertex shader to be actually considered by a fragment shader*. Hence the 'in' and 'out' variables have per-vertex values, whereas the 'uniform' has a given value for any vertex until updated. 'gl_Position' is a reserved name having the special meaning to the pipeline as that ( transformed ) vertex position in clip space ( as previously discussed ). Inconsistently, a fragment shader's output is an 'out vec3' or 'out vec4' of any name you like, the vector length typically depending on the presence/otherwise of an alpha channel.

[ 'gl_Position' is thus a special notation for a variable with 'out' functionality. One particular reason for that ( I think ) is that vertex position attributes must be rounded up into groups on a per-primitive basis later on. To properly clip for instance one must know what geometric object type is being worked upon. ]

- obviously we are doing a mat4 * vec4 -> vec4, assumed float numeric type of the array/vector elements. This being single precision IEEE-754 specification, or very close to if GLSL version is 110 or 120 ( possible hardware dependence ).

More later. ;-)

Cheers, Mike.

* To be precise the fragment shader deals with fragments ie. what the rasteriser has determined to be pixel candidates. The pipeline between vertex shader and rasteriser has a number of other transforms eg. clipping, perspective division and viewport. Thus there are also issues of magnification ( single fragment to many pixels ) and minification ( many fragments to a single pixel ) ....

( edit ) I've done "ready reckoner", "in a nutshell" and "for dummies" so far. I'll have to follow modern publishing trends and follow up with zingers like "Schrodinger's Shader", "Einstein's Screensaver", "Feynman Fragmented", "Fermi and the Pipeline" or "Dirac Meets the Transform". Fortunately cognitive dissonance will be quite unimportant. :-) :-)

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6542
Credit: 287066698
RAC: 97422

[mumbling to self] .... still

[mumbling to self] .... still working on stuff as regards providing a basic but useful abstraction of the pipeline. While trying to solve specifically what I want to do - the Starsphere rewrite & my SolarSystem screensaver - I also introduce some generality and room to vary/expand as well for down the track ( within the confines of OpenGL ES 2.x etc ... ). To that extent I am trialing the use of what are, for me, previously under-appreciated aspects of C++/OOP mechanisms eg. protected members. While protected data seems to be frowned upon ( sez even Bjarne himself ) it's deemed OK to carefully use protected methods - thus invoked by derived classes - to regularise the access to privately held data in base classes .... [/mumbling to self]

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6542
Credit: 287066698
RAC: 97422

Heartbeat : Pipeline

Heartbeat : Pipeline abstraction layer looks good, of course further testing will refine that view. Now replacing OGLFT by SDL TTF.

Cheers, Mike.

( edit ) And, at a glance, the latest SDL FFT does have kerning. Plus font foreground color can be altered per rendering call, without having to create a new font instance !! :-)

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6542
Credit: 287066698
RAC: 97422

Boner : I have a 64-bit

Boner : I have a 64-bit Ubuntu. I compile for 32-bit Linux. So that's a cross-compile, right? Yup it is, and I am a thick numpty. You need at least :

(a) the right gcc and ld switches, plus

(b) the right 32-bit libs, and

(c) the correct paths with your -I and -L options

So (a) thru (c) applies generically for all desired targets, with the addition of the right gcc version eg. i386-mingw32-gcc for win32 builds ( plus other target specific stuff ). The question then becomes whether that is already provided somewhere for your distro, or do you build the gcc version yourself ( which is purer, but a fiddly bear of a task ).

Of course, you all knew this .... :-)

Cheers, Mike.

( edit ) My machine is 64-bit because I have a glorious 32GB of main memory which just blitzes the builds, getting around a memory management issue with gcc when it has to expand some C++ STL code. You get a 2 minute build instead of twenty ....

( edit ) Although the -m32 gcc option will go a long way if your linux target is the same architecture as the linux build machine.

( edit )

Quote:
It would be hard to imagine why anyone would do the following : have all OpenGL features done on CPU code alone and simply write to a hardware framebuffer ( like the CPU address space mapped video memory of old ). But it could be done that way and be perfectly adherent to a given OpenGL specification.


I've recently found some linux mesa libraries which do precisely that. It's all software resolved on the CPU with a single bitblit to the video card once per frame.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6542
Credit: 287066698
RAC: 97422

Now here's a few things (

Now here's a few things ( what a learning curve, with much thanks to my son Stephen for actually nutting the problems out ):

(a) The 32bit build on the 64bit machine is now sorted, but was rather confounded by what follows :

(b) For the libraries that one builds from downloaded sources ( eg. XML2 ) then one may use a line like the following ( in the relevant makefile ) :

LIBS += $(shell $(PRODUCT_INSTALL)/bin/xml2-config --libs) that is, one asks the ( xml library writer's ) provided script to disclose what libraries are needed for xml2. Sensible. A library list as a lump is returned. However, per se, that doesn't allow separation of libraries into those that one desires as statically included vs those for dynamic referencing.

(c) It may well be that a given config script has not allowed for cross compiles. It may simply & implicitly inquire 'what system am I running on' and leave it at that. While one might have an expectation that the script ought respond to '--build=' and '--host=' command line switches, that is a convention and not necessarily honored. Hence one can't confidently convey any target system suggestion to the config script. So it might require forcing by other means eg. 'CPPFLAGS += -m32' say, in addition to outright choice of ( cross- ) compiler.

(d) In the case of SDL_ttf we have discovered inconsistencies b/w the following :

- the disclosed Freetype version per the webpage ( 1.2 ),

- the Freetype version included in the source download ( 2.4.12 ),

- the recommended Freetype version mentioned in the README ( not at all ),

- the Freetype version in provided binaries ( 2.4.8 ), and

- the Freetype version that actually may be preferred for other reasons ( 2.5.1 )

Or, if you like, demonstrating the importance of the cohesion within a developer's head, as well as b/w developer heads. :-) :-/

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6542
Credit: 287066698
RAC: 97422

OK, the screwy version

OK, the screwy version numbering logic continues ....

With OpenGL ES ( embedded systems ) there is different version numbering for GLSL ( graphics library shading language ) compared to plain OpenGL ( desktop ), indeed the shading language is known as 'GLSL ES' :

(A) for OpenGL ES version 2.0 select GLSL ES version 1.00 [ and that appears within the shader as '#version 100' as the very first statement, and this version will be assumed anyway if you omit to declare. ]

(B) for OpenGL ES version 3.0 select GLSL ES version 3.00 [ and that appears within the shader as '#version 300 es' as the very first statement, but version 100 will be assumed if you omit that. ]

I have no idea where "GLSL ES version 2.00" fits in ( it does exist ), but that's OK because I'm only only writing for case (A).

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6542
Credit: 287066698
RAC: 97422

Side note : because the

Side note : because the myriad of standards documents take no short time to trawl in order to answer even simple questions like "what shader language version do I declare given that I am using ES 2.0 ?", then the Khronos group have helpfully published some cheat sheets eg. this reference card. You could print that out and laminate say in hardcopy .... or just view on screen.

While I do use good book sources ( electronically via Safari Bookshelf ), there is a worrying tendency for those to rapidly date in some important details. So online references like this are marvelous to have handy.

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6542
Credit: 287066698
RAC: 97422

Side-note/trap-for-the-unwary

Side-note/trap-for-the-unwary : close inspection of the GLSL ES documents indicate the versions are effectively orthogonal. That is : there is no useful common subset of behaviours amongst them, so one doesn't have forward and/or backwards compatibility in any typical sense. Think 'forked'. This relates to whether the underlying hardware is the older 'fixed function' type, or the newer 'programmable' variety. For example, storage qualifier syntax :

GLSL ES 1.0 are none, const, attribute, uniform and varying.

GLSL ES 3.0 are none, const, in, centroid in, out, centroid out and uniform.

.... and thus in detail there is similiarity in semantics but not pure equivalence.

Hence there are rumors for the creation of compile time switches ie. version sensitive macros in the GLSL ES source code that permit conditional compilation ( occurring at runtime upon presentation to the OpenGL ES state machine ). There is even talk of creating an intermediate language akin to Java's Byte Code format !!

Hmmmmm .... it seems it wasn't foreseen that having discarded the abstractions of a high level graphics API ( pre OpenGL 3.1 ), and then morphing to a hardware abstraction layer for graphics card compute units ( post OpenGL 3.2 ) would produce much target hardware dependencies ( and so a plurality of approaches to device performance categories in the target set )? So now we 'zoom back out' ? Go figure ....

Cheers, Mike.

( edit ) Of course that does not prevent a 'programmable' variety behaving as if 'fixed function'. But one senses the inevitable future deprecation ..... I guess the underlying impetus is the pace of change of platforms ie. all those non-desktop handheld gadgets that abound these days.

( edit ) Fortunately this is not mortal for my 'framework' and 'ogl_utility' efforts. One simply alters the desired OpenGL type and major/minor versions to startup with in 'WindowManager::initialise()' - only three lines need to be altered. Then you load different shader sources ( eg. via ResourceFactory ) at runtime. I have used OOP to corral the change-sensitive areas !! :-)

( edit ) Actually I'm not so sure 'GLSL ES version 2.00' exists. I may have misled myself.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6542
Credit: 287066698
RAC: 97422

Alas & with regret I am going

Alas & with regret I am going to finally concede defeat upon this. :-(

It's just getting worse, the whole OpenGL thingy is now thoroughly balkanised as it were. Firm indications exist that the trend to have quite distinct/separate actual code specifications but commonly named for the sake of appearing to be coherent will continue. It is now a scattered flotilla of concepts with all the difficulties arising, as described, from the move away from backwards compatibility etc. Certainly for a lassez-faire developer ( I use the title loosely ) such as myself I certainly can't even grasp a stable collection to remain workable for even short time periods.

But what the heck! It's been fun and educational. Kept me out of the pub. If nothing else* I have learnt a metric tonne or two of stuff about 'requirements analysis' ! :-)

Darn but still Cheers, Mike.

* and that I otherwise stepped on nearly every mine in the field.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 947
Credit: 25167626
RAC: 13

Hey Mike, RE: Alas &

Message 78310 in response to (parent removed)

Hey Mike,

Quote:

Alas & with regret I am going to finally concede defeat upon this.
[...]
Bit what the heck! It's been fun and educational. Keeps me out of the pub. If nothing else I have learnt a metric tonne or two of stuff about 'requirements analysis' ! :-)

Darn but still Cheers, Mike.

Long time no see! What exactly are you referring to above? OpenGL ES or your whole >5-year (!) endeavour? I hope it's not about the latter - we still want to see an official/endorsed release of your "baby"! No offence to your son :-) Yes, sometimes the journey is more important than the actual goal but, hey, this baby deserves some exposure...

Best,
Oliver

 

Einstein@Home Project

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.