Is gprof ever works on os4 ? It is! And can be still!

@kas1e

Good catch. My guess is that those two are built from different revisions of the patch sets found here:

https://sourceforge.net/p/adtools/code/254/log/?path=%2Ftrunk

As you can see they aren't exactly small and the commit messages aren't very descriptive.

The question is who built the SDK:s, and if that someone can be found, does he know what revisions he used?

This is all guesswork though, perhaps the patches used are hidden on some defunct harddrive somewhere in a landfill in central Europe.

@kas1e

ld source code may be the same v2.18 ... just a small change to the include files can make huge diffrense on how it compiles and optimizes.. ie diffrent file size.

@sTix
At least we move somewhere :) My bet is that "ld" in "working" SDK and "non-working" sdk just have inbuild use of different ldscripts (because simple copy of ldscripts from one SDK to another didn't help). That can explain the different size of created test-binary and of gmon.out. All those ld-scripts are for pointing out to linker how binary need to be constructed: what sections, etc, etc. So very possible that there things go wrong.

And i think we need to take into action that no one will find anything, and we are on our own.

And good find in the history of Sourceforge! I am good at that kind of thing like building revision by revision and checking what happens (i.e. stupid, routine, time-consuming work). We may try to build one revision my one. And check what happens.

Or, we can try to see if there is any change in the commits about "ld-scripts" and related parts in the "ld" code.

For now, i will try to get out all from the verbose logs of "ld", to see if there are any differences. Maybe things will be that simple like different use of different ldscripts.


@rwo
Quote:

Yeah, i know... Need to find what exactly causes our gprof issues in all "ld" since SDK 51.3

@All

Soo... I just go verbose mode, and just do "ld -verbose", and that what we have:

ld from SDK 53.2:
https://kas1e.mikendezign.com/aos4/debug/gprof/ld_SDK_53_2.txt

ld from SDK 53.3:
https://kas1e.mikendezign.com/aos4/debug/gprof/ld_SDK_53_3.txt

What i can see there is that those things changes:

Line 16/17 with "PROVIDE". There start address changes. Before it was 0x0000000 when things work, then it start to be 0x100000000.

Also in "working" version we have 2 more sections in .text segment : .rela.got1 and .rela.got2 (see lines 49 and 50 in the 53.2 version).

Also line around 90-95, ALIGN string also have changed.

Then some other changes on lines from 150 to 160 about sections/segments/etc

And then lines 190-200 changes about ALIGN as well.

So that is what causes those things: binary changes. Start address, alignment, and some sections for sure. And as it all comes from LD-Scripts, i assume i can try to change it somehow manually to find out exactly what part is the guilty one, and then it will be easer to deal with.

@kas1e

Quote:

In amigaos.sh:

Quote:

Maybe, maybe.

@sTix
There i see "Date: 2010-05-12 21:19" , but things got broken in SDK 53.3, and all files inside 11/13/2008 (not sure though if we can trust those dates of course, maybe on Sourceforge they were reuploaded or something)

And next commit to what you point out set the new version for adtools: (adtools build 20100512)", while both ld binaries and working one and broken one, coming from SDKs both of which have "adtools build 20080706".

I also checked the dates of both "ld" binary from SDK 53.2 and 53.3: The working one was built 10/20/2008, and the non-working one was built 11/13/2008.

So just 23 days of commits should tell us something. Only to find the correct dates, as seems SourceForge ones a bit shifted seems so. Then if nothing is understandable from logs, i will try to compile version perversion to find out the guilty commit.

@sTix
I go commit per commit, to see when "adtools build 20080706" version changes in sourceforge repo, and the first one was :
commint 180:
https://sourceforge.net/p/adtools/code ... unk/gcc/gcc/version.c#l11


Then it was "20080706" till commit 231 included:
https://sourceforge.net/p/adtools/code ... unk/gcc/gcc/version.c#l11


So, between those 2 commits, we have 2 different SDK releases, in the first of which ld works, and after about a month - not.

The only question is if those commits contain aos4 changes at all. Or they just were applied after on top of something and those actual changes which were done for SDK are not here. Or they were applied just much later (already changed..).

@kas1e

True, no SDK:s come from the future. Almost only binutils changes between 180 and 231 it seems. No OS4 specific changes in binutils though. Some of it is obviously 68k, but I can't tell if there are generic Amiga changes in there as well.

@sTix
It looks like that before sba commit the things, os4 version of SDK comes from patches "hidden on some defunct harddrive somewhere in a landfill in central Europe". Changelog of them probabaly lost. Then, Sebastian just add what he got as last code from brothers and so already "changed" probabaly right after broken ld are in ..

I wrote mail to Sebastan with all details asking for info if he remember any or so, will see what he say.

But what is for sure, is that we can try just in current compiler try to mimic what working ld giving us, just for test, and if it works, then we can dig in deeper to those changes to know why/when/for_what and maybe can find the better solution.

@kas1e

Quote:

Let's hope Sebastian remembers something. But it's a long long time to remember things like this.

Digging into binutils is a major mental challenge, it's a really depressing place to be :)

@sTix
Another good progress!

I found that by default, "ld" has its own inbuild ldscript. I.e. just inside of "ld". And usually, that is the same script which we have in the SDK/gcc/ppc-amigaos/lib/ldsript/amigaos.x

You even can see in the binutils-cross-build-2.23.2/ld/eamigaos.c there are parts of that inbuild ldscript (search for "Default linker script").

Now, all that I do is just take the default "amigaos.x" ld-script from SDK 53.2 (when things work), and use that ld script manually with 53.3 SDK (when things do not work) and now even with the very latest SDK: and everything works! I mean with current LD etc. So issues id 100% in the default-inbuild linker script (or, an issue that things were forgotten to be changed in profile code of clib2, when internal ld-script were changed).

Through to provide manually ld-script a bit of pain, but for the tests more than enough, that how i compile it with working ld-script now:

Quote:

And newly generated gmon.out by -pg from latest/current GCC works! I can see everything.

Now, what we need, is alter that ld-script bit by bit, so to find out what string exactly causes the differences. My bet is still as i expect from the beginning: the size of binary and shifting of sections cause in some field of gmon.out at the begining have wrong size => issues. Once we find out what exactly, then things can be fixed easy.

@kas1e

Wow, fantastic. I admire your digging skills and patience :)

@kas1e
Awesome work Roman.
Those files are common in different gcc versions, right?

@sTix
Thanks, but that is only the beginning: now need to find what exactly causes issues, and how to fix them properly inside of the clib2 profiling code. But at least "bad routine" work done, and things start to be interesting now, yep.

Btw, if you check : https://github.com/sodero/clib2/blob/m ... brary/profile_gmon.c#L248

See there GetElfAttrsTags + GetSectionHeaderTags.. So constructions of gmon.out is definitely based on the structure of the binary itself and its fields. We only need to find what is wrong in terms of the current ldscript with that code.


@george
Quote:
Those ld-scripts come with the linker, yes, but they are different from SDK to SDK (depends on what was changed in the linker and related code, or when Binutils were changed, etc). And internal, inbuild, default linker-script also different depending on what was changed, etc.

GCC's Linker by itself it's just a command language taking linker scripts and building output binaries depending on the statements/etc in the scripts. Those scripts told how to build binaries in one or another situation (like default one, when -N options set, or -pie, or -r, etc, etc). Through default one, while coming separately in ldscripts directory, also inbuild inside of the linker too. Dunno why maybe to avoid setting it manually all the time.

Now what we found is that construction of the binary by ldscript from sdk 53.2 make generated gmon.out be correct one, while making a binary with all ld-scripts after that SDK build gmon.out which are bad. But all the linker scripts coming after were altered to have less size of binary, etc, etc, so just after some change to make a binary better gprof checking was missed by accident by SDK team probably, then no one uses it for some time, and once reports arise probably was too late to remember what can cause it.

For now, need to find what exactly changes in the ldscript started to cause that issue, and then, the clibs2 profiling code needs to be changed/fixed (depends on what we found and what exactly cause this issue). And then newlib too need to be adapted accordingly.

@sTix
Found, line 11 of SDK/gcc/ppc-amigaos/lib/ldsript/amigaos.x. The working was one when there were 0x00000000 in 2 places, and it did not work when it 0x01000000 in 2 places.

I just now take GCC 8.3.0, grab ldscript from, change only this line, rebuild with manually specific this ld-script and gprof are fine with.

Now to understand what is wrong with the code in gmon_out.c of clib2 profile, that elfheader data return something bad that broken gmon parsing.

@kas1e

In revision 267 TEXT_START_ADDR=0x01000000 was added to binutils/ld/emulparams/amigaos.sh. Before there was no such entry, which maybe, perhaps, maybe results in a default value of 0x00000000.

In binutils/ld/emulparams/amigaos.sh TEXT_START_ADDR is explicitly set to 0x00000000.

The commit message in rev 267 and 265 says that the changes come from Hans-Joerg and Thomas Frieden. On other words the changes might be older than what the SVN log indicates which means that it might be included in the SDK from 2008. This is pure speculation though.

@All
So, to make things short, if anyone wants _RIGHT NOW_ to use gprof for profiling their amigaos4 binaries, you need

1). Have a machine with "performance monitor" working (Pegasos2, Eyetech's Amigaones and Classics. In beta also x5000 and Tabor, and soon should be x1000 as well)

2). In your SDK find the main "ld" binary (on native SDK installation that is in SDK:gcc/bin/ld, on cross-compiler setup that is /usr/local/amiga/bin/ppc-amigaos-ld.exe). Make a copy. Then in hex-editor (i use Zaphod), find the words "Default linker script, for normal executables" that will be only in one place. Next, you can see after about 160 bytes this string: "PROVIDE (__executable_start = 0x01000000); . = 0x01000000 + SIZEOF_HEADERS;". There you change in 2 places 0x01000000 on 0x0000000 i.e. you just delete "1" in addresses and put "0" instead). Now, every binary you will be compiled with those changes (binary will work, but how bad is it to have that change in the system meaning that unknown, but binary still works, and you can profile it). So you can have 2 copies of "ld" binary, one when you just need profiling working, and another one when you just need the "usual" binary.

3). After you patch LD you then build your binary with "-pg" and "-gstabs". You can use _only_ clib2 build as newlib's profiling code seems wasn't implemented properly (even the tests with old working SDK 53.2 from year 2008 show that and it is the same now as well).

To build clib2 build will be:

Quote:

4). It will create "gmon.out" file for you after you exit from your program, and then you just run "gprof name_of_binary".

Gprof can be used of any version (even the latest one from GCC 11.x). And SDK can be used the latest one and GCC too, just with the latest GCC you will have warning errors from gprof that "dwarf5 is not supported, only 2,4,5" but that harmless, profiling info is still there.


Now to raise the interest, if you on cross-compiler, you can use some python script called gprof2dot.py from https://github.com/jrfonseca/gprof2dot. The purpose of that script is to convert the output from many profilers into a dot graph (including gprof, of course). For that script to be used you need the "dot" command which is part of Graphviz packages (which I do not know if we have natively, but pure "dot" command for sure possible to port natively, so after a bit of work that can be used natively as well).

It can make from your profiling info all kinds of graphs, but i love colorful ones. Check some examples to see how important for us to have gprof working:

basic test case:
https://www.thegeekstuff.com/2012/08/gprof-tutorial/


output:
https://kas1e.mikendezign.com/aos4/debug/gprof/test_gprof_clib2.txt

graph (press open in new tab for full size):


Tequila from Capehill (yeah, profile the profiler) :
https://github.com/capehill/tequila


output:
https://kas1e.mikendezign.com/aos4/debug/gprof/tequila.txt

graph (press open in new tab for full size):


Horny app from Timo (that big one, yeah, binary 1.5 mb of size):
https://github.com/kas1e/midi-sequencer-amigaos


output:
https://kas1e.mikendezign.com/aos4/debug/gprof/horny.txt

graph (press open in new tab for full size):



Btw, from graphs, you can see straight away that in Horny not the main() is the "main" body used, but some "haupt()" function, which if you check the source code of Horny is it (so at least that part our profiler from clib2 take surely correct).

So.. What else needs to be done for "proper" gprof support:

1). Understand what is all about 0x01000000 vs 0x0000000 and fix it properly does not matter where it needs to be done.

2). Fix in X5000/Tabor performance monitor so clib2 profiler will work there too

3). Fix newlib profiler to make it work as expected (at least as in clib2 at minimum)

So that is not the end for now, but at least everyone who needs gprof right now can use it if have a necessary machine.

@kas1e

Very nice. What happens if you build with the unpatched ld and add '-Ttext=0x0000000' when building?

What is the output from the following 3 commands:

> nm file_built_with_patched_ld
> nm file_built_with_normal_ld
> nm file_built_with_normal_ld_plus_text_option

@sTix
Quote:

Works too ! I just do that:
ppc-amigaos-gcc -mcrt=clib2 -Ttext=0x0000000 -pg -gstabs test_gprof.c test_gprof_new.c -o test_gprof_clib2 -lprofile


And it works!


Quote:





Quote:



Quote:

@kas1e
Quote:

Brilliant. I guess you could ask the original authors if there's a good reason for having text at 0x01000000 and not at 0x00000000? Are the Friedens on a beta-tester mailing list of some sort?

But the output from 'nm file_built_with_normal_ld' seems identical to 'nm file_built_with_patched_ld'?