GL4ES: another OpenGL over OpenGLES2 emulation - some tech. info and porting progress

@thellier
Quote:

We use much more comples shaders already with nova (with structs and all things), strange why it fail with such a simple one at moment. But it should be soon easy to find the roots imho, as it already really small.


Quote:

Do you mean copy of fog_struct things to the fresh uniforms and use them in both vertex and fragmet , like this ?:

Vertex:



Fragment:



So to not use "vayring of struct" ?

No i was meaning: fully remove the struct container for fog and just let the the fog variables as is

@Theiller

Get rid of whole struct and use fog variables without will be harder, as gl4es code expect structure to be there ..

So instead i just for sake of test do a copy of all entries from fog's stuct, to some fresh varying values. Like that:

Vertex:




Fragment:



And fog works fine ! Check screenshot (press open in new tab for fullsize) : on left side gl4es version with those shaders, on right side minigl one:




That mean, that probabaly as Daniel expect it can be ogles2 issue : data in vertex shader in structure are correct. Just when we "uniform" it between two shaders then it fail.

It also mean , that we was lucky to have found that issue only now. I imagine that in all other places it all can be wrong where structs used as uniforms. For example Lighting. And issue we have with lighting for example in FrickingShark (when things too dark) can be very possible because of that.

Don't know if it was there since begining , or some new issue through.

>struct and use fog variables without will be harder
No because in fact it is the same thing
I mean
struct _gl4es_FogParameters {
lowp vec4 color;
mediump float density;
mediump float start;
mediump float end;
mediump float scale;
};
just say that there a 8 floats at this offset inside the uniforms data
replacing it with
lowp vec4 color;
mediump float density;
mediump float start;
mediump float end;
mediump float scale;
change nothing to the way uniforms are used (same data same usage)

Anyway as you said it seems the problem is transmitting the uniforms value from vert shader to frag shader

Have you noted that vert shader and frag shader dont use the same uniforms variables ?

I mean there is at uniforms' start
uniform highp mat4 _gl4es_ModelViewMatrix;
uniform highp mat4 _gl4es_ModelViewProjectionMatrix;
in vert shader not in frag

It is allowed to have differents uniforms for a vert and frag shader BUUT does the prog that use those shader REALLY change the uniforms when calling frag shader ?

@thellier
Quote:

I not sure if it about any uniforms values with the same name, or only about uniform structs values with the same name. Daniel explain me a bit about how things done inside of ogles2 when code happens to have 2 uniforms with the same name : physically its not the same uniform, but 2 separate ones, and ogles2 writes 2 uniforms one to vertex and one to fragment one. So it looks like second write (to vertex) may fail.

At least, the symptoms we have and tests with varyings, prove that shaders recieve uniforms, but correct one we have only in vertex one, but not in fragment one. That why Daniel think its probabaly ogles2 (we at least hope, as if so, then Daniel can deal with it in next few days. Because if it Nova, then buy-buy, wayting for no-one-know-how-long).


Quote:

Yeah of course, that happens to be in shaders, because they auto-generated by gl4es, so sometime some values are declared, but not used (so to cover all the most cases of shaders need it to emulate all those fixedpipeline things). Sometime they used only in vertex, or only in fragment. ptitSeb tried hard to make generated shaders be good as possible, but there still can be places when some unnecessary declaration happens, or something of that sort (which of course, should't do any harm).

But in our last case, we share only one uniform : fog's struct, other uniforms declared in vertex shader used in vertex shader only.


Not sure if i understand well what you say there, but imho its nothing unusuall when things used only in Vertex shader and not in Fragment one ?

For example sometime we even have fully empty gragment shaders, just main() {} , when for example game want to fill z-buffer only (for multi-pass rendering , etc). Or when game, want empty fragment shaderfor "update clip region" operation (for performance reasons, when coders wish to do so within a single subpass, and don't want the "update clip region" operation to touch the color attachment (e.g. no blending of fully-transparent fragments into the framebuffer)).

But having fragment shader without uniforms which have vertex shader, or even have empty fragment shader fully, should't mean that it should't work, right ?

I was meaning we need to have to 2 calls to DBOSetBufferTags
1) to set the uniforms for vert shader with a dbo (=uniformes data) that contain the 2 matrices + the fog values
2) ... frag ... With a dbo that contain only the fog values so with a smaller size

@thellier
Quote:

Dunno..

As i understand Daniel's answer, its anyway 2 "different" (physically) calls with "different" uniforms, even if they have the same name, but also of course if they didn't have the same name. As Daniel explain me before , when we do glUniform, then ogles2 searches in both vertex and frag shader for a corresponding variable location and issues a DBO write if necessary. So probabaly its already splitted between enough (taken in account that even if there is uniforms with the same name, they anyway 2 different ones, with different dbosetbuffer(), etc, then probably when they differs, its then of course will be different calls).

@thellier

Btw, i re-read topic back to the page where i found issues in fricking shark's shaders with lighting. There:

http://www.amigans.net/modules/xforum ... id=115343#forumpost115343

See, boch shaders there have "uniform int g_ActiveLights;", and issues we walking-around was something with exactly that "g_ActiveLights" value.

But in this case its not struct, just "2 times the same uniform in both shaders". So... probably the same issue , and not related to struct, but just to "2 uniforms with the same name in both shaders".

And before we even didn't tried to copy those uniforms to fresh varying from vertex so to tranfer them to fragment (for sake of tests, as we do it now at last). All we do back in past its checking if g_ActiveLights send to vertex shader correctly : and yeah, it was send correctly. But then if we have there the same issue, then of course fragment one didn't got correct one, and things not works as expected and produce those "all darker, all black" effects. Interesting! Hope fixing that issue will fix few issues at one push :)

You and I and Daytona agree : it should be done this way (2 dbo + 2 dbosetbuffertags

But does it really happen ?

Can we track down the calls to dbosetbuffertags ?

Quote:

glSnoop can track now all warp3dnova and all ogles2 calls, as well as you can choice only to trace calls you need. So, i add just W3DN_DBOSetBuffer there, and do 2 traces:

1. with non_patched version of shaders (i.e. original, where we have our issue):

http://kas1e.mikendezign.com/aos4/gl4 ... g/non_patched_version.txt


2. patched version with varrying (where we get rid of issue):

http://kas1e.mikendezign.com/aos4/gl4es/fog/patched_shaders.txt

As i can see , we have in both cases 2 different DBOs, so call is happens. Addresses also sane all the time : one address seems for vertex dbos, and another address seems for fragment dbos.

The only difference i can see between too : its value of tags. In non_patched version is 0x60f9f2cc , and in patched one is 0x6319029c. But that probably also ok as address changes from run to run same as address of databuffer.

Does the 2 dbo sizes are différent and are what we expects ? I mean a size for 2 matrices and fog values for vert and a size for only fog values for frag ?

@thellier
Quote:
160 for vert and 32 for frag

Quote:

At least frag one are 32 , which mean 8 floats, which are right for fog struct (4 fog floats, and vec4 also 4 floats for color )

With vertex one we have no problems, but probably it correct too : 2 mat per 64 bytes so 128, + 32 for fog struct = 160.

Seems that just content of DBO for fragment one are wrong and not the same as for vertex.

Not sure if i can dump content of each dbo to compare.

@kas1e
@thellier

When I wrote kas1e it was during holidays far away from any machine and without being tooo exact So if there's any confusion, let's clear it up:

Of course it's always about ONE (1) DBO. You only can bind 1 DBO just like you can only bind 1 VBO to a shader pipeline.
But this single DBO is partitioned into TWO memory sections, one for the vertex shader and one for the fragment shader.

If a uniform with the same name appears in both shaders then physically those are NOT the same and ogles2 has to detect that situation and issue 2 writes into the DBO: one in the area for the vertex shader and one into the area for the fragment shader. That's simply because ogles2 doesn't distinguish between the vertex- and the fragment-part of a shader program when it comes to glUniform.

That's actually the only special thing ogles2 has to do. And so far I didn't find a bug on my side here. So far I didn't see any reason why ogles2 should only update the vertex shader area and not both, which is what the symptoms look like.
But I'm still checking

Internally things get a bit more complicated because for performance reasons DBOs are mirrored and multibuffered similar to what I do for VBOs.

@All

Ok, good news : thanks to Daniel issue with those struct uniforms was fixed ! Fog works now everywhere where it didn't before. He will explain all stuff tomorrow, but hey hey , one bug less !:)

@all
Fog-issue: it was a bug inside ogles2.library, found and fixed it yesterday Has nothing to do with the struct itself.

A bug in the uniform shader variable handler resulted in same-named (and thus in the context of a GLSL GPU program logically identical) vertex- and fragment-uniforms to eventually be assigned different location numbers.
If the user later issued a glUniform(location_number,xyz) call then the lib would only find and set the vertex-shader's uniform and skip the writing to the fragment-shader's uniform because it would simply not find it (unless you used expclicit location specifiers in the GLSL code or were lucky)

OpenGL doesn't distinguish between such same-named but physically different uniform variables in the shaders of a GPU program - in stark contrast to Nova, where all that info is shader and not shader-program based. Consequently Nova's ShaderGetObjectInfo, which is used to query uniform variable information, works on shaders only.
Also, Nova only gives us a GLSL / GL compatible "location" info if it has been explicitely defined inside the shader, otherwise it doesn't generate one but only returns W3DN_NOLOCATION.

Because of all this ogles2.lib has to generate such location infos by itself at program link time, in a GL compatible way. Which means that in case of a same-named variable in both of a program's shaders the same uniform location has to be enforced -> which is where the flaw was

@all

This is still an issue.
(ScummVM's launcher displays a garbled background in launcher when there is an in-app message shown)

Should i file a report somewhere so it doesn't get lost?

@Raziel

Have you tried my experiment, disable FBO in context.cpp? It cured the 1st frame issue but after that I haven't had time to re-debug it.

@Capehill

Nope, will check tomorrow.
Rereading your post I thought you were talking about the gl4es source and not scummvm, my bad

@Raziel

Try modifying this line to false https://github.com/scummvm/scummvm/blo ... s/opengl/context.cpp#L155

How about the GLUT Library? GL4ES comes only with GLU Library. Mixing with the GLUT Library from MiniGL seems not possible.