GL4ES: another OpenGL over OpenGLES2 emulation - some tech. info and porting progress

@Daniel
Some good and bad news on crash front about FBO in supertuxkart:

That broken look of cars when we use LIBGL_RECYCLEFBO env was indeed because of gl4es, but then, once ptitSeb fix it, and it start to show cars correcly, it start now crashes the same as without LIBGL_RECYCLEFBO ! Damn :( With the same:

[error] Irrlicht: FBO has one or serveral incomplete image attachements
[error] Irrlicht: FBO error

So the bad news that we didn't find roots of crash, and the good news, that it some general issue, which is NOT fixed by usage of LIBGL_RECYCLEFBO, and roots of crash somewhere else and more general just than regrouping/reusing fbos..

@kas1e
Then I'll have to inspect it in depth to find the cause and whether it's related to ogles2 or Nova or whatever. Please prepare a nice package for me and upload that to my FTP server, as usual Also, if you have fresh trace-logs with the latest glsnoop, hand'em over please!

@Daniel
All done !

game:
reports_here/supertuxkart.lha

full trace/profile from start to crash:
reports_here/supertuxkart_tracelog.txt

Through to note : when you start it, it will take about 100 seconds to load till menu (loading screen will appears with some progress, but still).

I hope to reduce loading speed if possible somehow (in snoopy log i can see lots of GetFilePosition/ChangeFilePosition , which probabaly mean fseek() / ftell() / etc, which are that slow on our side by some reassons). Not so matter now anyway, just saying so you know how it should be for now.

So just unpack, run, wait 100 seconds, then choice "single player" , and try to move mouse cursor over cars. It can crash immediately, or after 2-3 cars (random luck).

Sometime, in very rare cases, you are able to choice car and start play, but it just 1:5 times.

After ptitSeb fix rendering with LIBGL_RECYCLEFBO and we start to have same crashes as without that environment, ptitSeb bring idea that maybe FBO needs some pause to be dealt with, and as test we do flush of all GL operation before attaching a texture to the FBO : but that sadly didn't help and crash still here (but that probabaly also reduce a bit route to search on our side ?)

@Daniel
Oh, found i way how to make for you faster loading , without needs to wait 100 seconds (by faster i mean 10-15 seconds instead of 100) : just remove directories "data/karts/konqi" and "data/karts/sara". First one use 19mb .b3d file, and second one 8mb .b3d file, and seems that our realisation of ftell() in newlib, suck some amiga1200, and very slow. At least in the snoopy log i can see a loooot of those GetFilePosition()/ChangeFilePosition() calls, which probabaly mean newlib's ftell().

In other words, remove those 2 directories, and loading will be 10-15 seconds, very good for tests.

@Daniel
Hate to bring your attention back to some old bug (probabaly), but i think we didn't fix fully issue with which we fight times back in Lugaru and which you fix in 2019-03-30 called "one of the 32bit-step-hashers had a problem with small data-packets".

So while Lugaru is correct now, i firstly found the same (i hope its the same) issue in one place of new SuperTuxKart. I didn't take an attention to that at first, as it happens vere rare, so i think at first that maybe something with STK's code.

But then, today i notice very clear that issue in the RTCW-Reboorn too, so seems that issue indeed there. I also very carefull check STK and yeah, it have the same issue too.

See screenshots (all taken from ogles 2.10 (09/19/2019) , press open in new tab for fullsize):

1. RTCW-Reboorn (see sky in the middle, remind issue in Lugaru skybox we had before):





2. SuperTuxKart (see vertical line in the middle of sky on all screenshots):








Should to note, that broken geometry not visibly very well as it was when we fight with Lugaru's sky-box issues when it was all the time right in the face. Now it happens very rare, but still happens as can be seen.

@kas1e
Regarding the FBO issue: I found that the respective FBO becomes "incomplete". However, the parameters seem to be correct, I still didn't find out what's causing this.

Regarding the skybox issue you just brought up:
from the screenshots it looks as if there were two different issues. Two of the Tuxkart screenshots look like the hash problem reappearing indeed. However, to me the other skyboxes seem to be correct regarding the geometry (the clouds' edges seem to match) but only the coloring seems to be wrong. But this can be coincidence as well.
I've put a test-lib on my ftp for you, please check out if that makes the problems disappear.

@Daniel
Oh i was too fast ! I tested that version of supertuxkart on win32, and that vertical line is here ! Check this out:



So at least with STK, its or problems in STK itself, or, what very unlucky, and in win32 and in our opengl driver, which is imho unpossible :) So imho that STK. But i anyway test it with no-hash version of library, and issue still here as expected. So imho that one we can no worry about, as if it on win32 too, then..


As with RTCW-Reboorn : tested no-hash version of library with that game as well, and bug still there too.

But now, after STK one was in the STK itself (i think it very unlucky that its win32 drivers too), i start to fear that maybe issue with RTCW-Reboorn also the game's code too..

Will try to setup win32 version of Reboorn, to see how it there.

@Daniel
Well, that was another unexpected (or expected :) ) thing, but RTCW Reboorn bug is also not on our side, but also on win32 ! I.e. that the game's code bug as with SuperTuxKart, see:



And that not all. Remember those issues with "Dots on the wall" sometimes ? There is win32 version of it:





And that not all ! Remember issue with "blink triangles overwrite each others in some place" ? There is win32 version of it too:



Damny crap it mean ALL those issues in RTCW-Reboorn are games code issues, and not our ones !

And that is good news : our drivers are fine in that terms :)

I also subscribe to those guys who make this Reboorn, they release privately some patches from time to time, will see if / when they fix all those issues , and probabaly after Huno can build new version of it.

But anyway, how relief is it : all bugs in RTCW Reboorn is not our faults

@kas1e
(leaning back, putting feet up) Wow, that's fantastic news - at least for us Once more: thanks for all that (re-)testing!

@All
In light of some hope that there will be warp3dnova fixes, working on another game, and find out, that seems some Fog shader didn't works as expected.

At least, i _think_ it can be shaders, but can be that something also in the glFog/glFogi, dunno.

So, that what we have visually when we compile the same kind of simple sdl1/opengl test code (press open in new tab for fullsize images):

1. Win32 / OpenGL




2. Linux / Mesa



3. Linux / GL4ES



4. AmigaOS4 / MiniGL



5. AmigaOS4 / GL4ES (bah!)




As can be seen, GL4ES are not guilty : on linux same code works. Whole code also not guilty : it works everywhere. Even on MiniGL it works, which mean that "fog" itself can works on warp3dnova (as minigl i test works over warp3dnova, but minigl didn't use shaders while working over warp3dnova). But gl4es version on amigaos4 are use shaders of course, and fail visually. So probabaly because of shaders (or, if something with glFog/etc , which is probabaly not the case).

Now, there is 2 shaders we have generated for that test case:

Vertex one:



Fragment one:




And there is full archve with source code of test case, and binaries for all platforms:

http://kas1e.mikendezign.com/aos4/gl4es/fog_test.zip

Any ideas ? Maybe someone can see from the code and look of something which may cause that ? Visually as can be seen there like no fog working at all.

I may try to reduce test cose more-and-more to simplify shaders till the moment while they still didn't works as expected, but maybe someone may see something already or have an idea about.

I was able to reduce init() part to just that (to still see that it works on win32 & linux/gl4es and didnt on amigaos4/gl4es), by disabling more fog parameters and lighting:



So, shaders are now:

vertex one:



Fragment one:



So even with that simple ones bug stil lthere

And now we replaced mix() from fragment shader on non-mix version doing the same, as well as add setting of alpha after (just in case it looses somewhere), as well as remove from the structs unused fields (just in case too), and now shaders are really small ones, and still give that bug:

Vertex one:



Fragment one:



What it can be ?:) Where ? abs() and clamp() probabaly all should be fine as they basic ?

Hello

1) Have you tried to let the Vertex shader unchanged and just let something like that in the Frag shader's main()

gl_FragColor=vec4(FogF,FogF,FogF,1.0);

So we will see if vertex shader generate a good FogF value

2) then try
gl_FragColor=_gl4es_Fog.color;
to see if the uniforms' fog colour is really send to the shaders

3) then try
gl_FragColor=vec4( _gl4es_Fog.density ,_gl4es_Fog.density ,_gl4es_Fog.density ,1.0);
to see if the uniforms' fog density is really send to the shaders

Note: putting the value to debug in gl_FragColor is a good way to find a shader problem as it can be read in the resulting picture in the RGB pixel value

@thellier

I tried for sake of getting rid of exp() thing in vertex shader Linear fog type, and then your first test. So shaders now looks like this:

Vertex one:



Fragment one:




Bug still there. Visually nothing changes in compare with original shaders.

So it seems that its vertex shader is not generate a good FogF value, right ? And there is no point to check 2) and 3) then , but something in vertex shader instead ?

>And there is no point to check 2) and 3)
I dont agree: perhaps _gl4es_Fog.density & _gl4es_Fog.color are not correctly transmited to BOTH shaders but you can only display it with frag shader with gl_FragColor=_gl4es_Fog.color;

BTW are you sure the "mediump" are needed in struct _gl4es_FogParameters ?

Quote:

Done all 3 checks then (press open image in fullscreen for fullsize):

1) gl_FragColor=vec4(FogF,FogF,FogF,1.0);



2) gl_FragColor=_gl4es_Fog.color;



3). gl_FragColor=vec4( _gl4es_Fog.density ,_gl4es_Fog.density ,_gl4es_Fog.density ,1.0);



The only differences i see visually its when 1) is used. Cube at least have changes something and not solid black. While of course still wrong, as it should be just like this:




Any more ideas ?:)

Maybe data transfert between vertex shader and fragment shader goes wrong (the "varying" process) ? But then, we already use in gl4es "varying" a lot as well, as it generate far more complex shaders for other games (like neverball, frickingshark, etc)

@all
So as next check i doing some thing Daniel have idea about: To check if the data arrived correctly in the vertext shader.

So we do the following: Instead of using fogstruct.color in the frag shader, transfer the color from the struct into a fresh vec4 varying in the vertex shader and use that one in the frag shader. If the correct color shows up then Novas varying handling is broken.

I.e. like that:

Vertex:



Fragment:



So, it show me color of cube the same as background. Probably that what Daniel expect it to be by meaning "if it show the correct color".

And i can even change the color to one i need, like:

testV = _gl4es_Fog.color;
testV.g = 0.3;

And it show correctly changes.

Then, i going back to original shaders to check that final FogF value in Vertex.

What i do, is instead of:

FogF = clamp((gl_Fog.end - fog_c) * gl_Fog.scale, 0., 1.);

i doing just:

FogF = 0.1;

So, i can see that "FogF" value works in fragment shader, and with that 0.1, it show me what it should show for 0.1 (almost the same color of cube as background, all fine then).

But once i add there anything from the gl_Fog. structure, then i have black cube. For example, just if i do something like this:

FogF = 0.1 - fog_c;

then cube is Black.

if just:

FogF = 0.1 + fog_c;

then cube is fully white.

Not sure what happens , but probabaly all of this mean that there is issue with struct varying. And maybe that is somehow related to issue we have with FrickingShark's shaders few pages back. FrickingShark's ones wasn't about "varying struct" of course, but it do have "varying of modelmatrixes" too, so cross the fingers its all can be somehow related.

Ho Roman,

Is your code correct?
Fog_c : why use a sqrt on a squared number ( Sqrt of x*x)? And not x directly ? Or should it be something else?

Regards.

@Petrol
It is for sake of tests , to get rid of abs() (to be sure that is not abs() are buggy in Nova).

2) gl_FragColor=_gl4es_Fog.color;

all black so the fog color is not transmited = perhaps using a struct for fog params is wrong with Nova

So transmit fog params as basic uniforms in both frag & vertex shader

uniform vec4 Fog_color;
uniform float Fog_density;
uniform float Fog_start;
uniform float Fog_end;
uniform float Fog_scale;