Brilliant. I guess you could ask the original authors if there's a good reason for having text at 0x01000000 and not at 0x00000000? Are the Friedens on a beta-tester mailing list of some sort?
Yeah, wrote mail to beta list already
Quote:
But the output from 'nm file_built_with_normal_ld' seems identical to 'nm file_built_with_patched_ld'?
Yes. The file size is the same too. The only difference is that in the many places of the binary in the original version of ld we have 01 at many places, while with patched ld we have 00 on the same places.
When we use original ld with adding -Ttext=0x00000000 , then the size of binary start to be much bigger (147kb instead of 83kb), but still provide necessary information for gmon.out to make gprof work.
@sTix We need to keep in mind that it was in 2008, maybe at this time, everything was too raw. Now it's 0x0100000, and probably simply changing 0 there to 0x01000000 will fix everything too... But that surely is it.
I just normally understand the format of gmon.out, and it's not a.out, but own format. There 3 sections in the file:
1. Header:
struct gmonhdr {
uint32 lpc; /* base pc address of sample buffer */
uint32 hpc; /* max pc address of sampled buffer */
int ncnt; /* size of sample buffer (plus this header) */
int version; / version number */
int profrate; /* profiling clock rate */
int spare[3]; /* reserved */
};
So it 2 addresses: lpc (low-pc (start address) ), and hpc (high-pc (end address) ), and all other are values and whole header takes exactly 32 bytes.
Interesting, that profrate for us is 100. While for example, i read in some docs about arm-cortex-m, they use 1000. But maybe that is due to fact that old classic machines are supported too, where 1000 may be too much, dunno.
2. Then come "kcount" block.
It's the histogram of the sampled PCs (Program Counter). The size of it (size of the whole histogram) varies of course depending on the binary and calculated inside of the profiling code/library.
3. And at the end of gmon.out we have 3st block : rawarc:
That one struct with 2 pointers (to the calling site and * the called site) and a count (that struct i have not learned at the moment, but that is not so important for now).
So, as you can see all vary depending on the start/end addresses of the binary.
Maybe that 0 in "lowpc" in 2 places there because originally, in 2008, the start address was 0x00000000, profiling code was written, etc, all fine. Why hardcore: maybe calculations with zero addresses all lead to some NULLs or so, dunno, and so were hardcoded?
I think i need to try now to just grab clib2 profiler code, and build my own one, where play with things.
EDIT: Btw, I also found 10-year-old mails from Frank Wille where he explain to me some things about 0x0100000 which I didn't understand back then or completely forgot, so, at first he think that:
Quote:
As you see, even OS4 executables are linked to being executed at an absolute address of 0x1000000. But this is only faked because the relocations will recalculate all absolute addresses in the program before it executes.
Without relocations (if the ELF loader allows it, don't know) each new process would be loaded at 0x1000000 and overwrites the previous one.
But then he checked it more carefully, and:
Quote:
I checked it again, and what I told you was not 100% correct.
The ELF loader seems to ignore the load address of 0x1000000+size_of_headers from the executable completely.
It just allocates some free memory and loads the program segment there.
But it doesn't relocate all the absolute addresses within the program code and data, because this information is missing without "-q".
When i ask after all that Frank is "Then what the reasons to have that 0x01000000 address mentioned in the ldscripts at all ?" i was told that "It is just a dummy, a placeholder. Maybe they didn't want to use 0x00000000."
@sTix I just grab those 3 files used in the clib2 profiler and build my own libprofilka.a , so can fast change/check things.
So, if in those 2 places where we have 0 and where was before p->lowpc (probably original from another platform) i put back p->lowpc and build with unpatched ld, then gprof says that created gmon.out in not gmon format :) And the same if i put 0x01000000.
So those values are used somewhere somehow for something else in the profiler's code probably. For real libprofile.a it just 2 C files a 1 asm file (for counts, can be used as it for our tests). And 1 more profile_profile.c file, which is part of libc, so to include profile code in.
EDIT:
Well right, we have:
frompc = 0x01000000; /* on os4 original was : 0 , with such comment: FIXME: was p->lowpc; needs to be 0 and assumes -Ttext=0 on compile. Better idea? */
frompc += fromindex * p->hashfraction * sizeof (*p->froms);
Maybe issue is coming from the fact that "lowpc" (i.e. base address) is somehow weird/amiga-specific, see in profile_gmon.c:
for (i = 0; i < numSections; i++)
{
shdr = GetSectionHeaderTags(elfHandle,
GST_SectionIndex, i,
TAG_DONE);
if (shdr && (shdr->sh_flags & SWF_EXECINSTR))
{
uint32 base = (uint32)GetSectionTags(elfHandle,
GST_SectionIndex, i,
TAG_DONE);
*lowpc = base;
*highpc = base + shdr->sh_size;
break;
}
}
So lowpc taken from there, and then by some reasons can't be used as on other oses (as for example i point out above).
Edited by kas1e on 2022/1/23 21:53:54 Edited by kas1e on 2022/1/23 21:54:34
@sTix I got an answer from Thomas (but it's not him who wrote all that parts about "0" and "FIXME", and he dunno who it was, he made some initial work on this, but not the code we have now), and also asked Frank if he understands wtf (he help me often with all hardcore crap, even if he is not in the os4 dev team) and there some tasty information i got.
For first, Thomas thinks that 0x01000000 switch was necessary for the shared objects support since these did add a .plt section, potentially in front of the .text (which means 0 as text base address was no longer working). So that kind of explains it. He also thinks it should be enough to add the base address (0x01000000) to any sampled address it finds, because right now, when sampling, it probably just subtracts the binaries load address, so it would need to add the base address at that point too. That can explain why a simple change to 0x01000000 didn't help in the profiler's code.
Also, he checked gprof implementation and find that it implements a symbol table for the binary in question. And since we need to translate from the actual LoadSeg() address to .text segment addresses, the symbol table should take care of the translation.
In other words, what Thomas means, is that we need to translate from LoadSeg() address we got by GetSectionTags to .text segment addresses (i.e. our "lowpc" which currently we set as 0).
Then, Frank says that yes, the 0x01000000 is arbitrarily chosen and has no real meaning, as the executables will be relocated to their real load address in any case. He also thinks that we have a bug with all that 0x00000000 because even Unix ELF executables will always have a non-null start address as well.
He also checked the code of the clib2 profiler briefly, but he also thinks that the key is profile_gmon.c, which uses a hard-coded lowpc of 0 in several cases. I.e. those "FIX me" and stuff. He guesses that monstartup() is called with low_pc and high_pc zeroed, so mongetpcs() must be called, which fetches the start- and end-address of the first executable section (.text) from the ELF file. There is certainly reads a low_pc of 0x01000000, which is in conflict with the zero hardcoding.
And probably newlib's code was more or less a straight copy of clib2 profiler (or the other way around, dunno) and have the same hardcoded lowpc to zeroes, and that then explain why it fails the same.
Now we only have to somehow fix it. Anyone ?:)
If anybody is willing to fix it on payment basic I also will be happy to pay a little for it.
For first, Thomas thinks that 0x01000000 switch was necessary for the shared objects support since these did add a .plt section, potentially in front of the .text (which means 0 as text base address was no longer working). So that kind of explains it.
Sounds reasonable.
Quote:
Then, Frank says that yes, the 0x01000000 is arbitrarily chosen and has no real meaning, as the executables will be relocated to their real load address in any case. He also thinks that we have a bug with all that 0x00000000 because even Unix ELF executables will always have a non-null start address as well.
I'm not sure that we can compare with Unix ELF executables that very often aren't PIE (or at least haven't been in the past). And if not relocated, it's probably wise to avoid an address of 0.
About fixing it, it's better if someone with a working performance monitor does it
@sTix Right, so will try to find out someone who can do so. But probably it's only Corto, Thomas, Salas00, or SBA1. We can start from 1000$, and I can cover it by myself in general if we will find someone.
@All In meantime, i just take the clib2 libprofile code, and made it independent so you can just build it without touching adtools/SDK, and use it instead on linking. There are archive:
Right, so will try to find out someone who can do so. But probably it's only Corto, Thomas, Salas00, or SBA1. We can start from 1000$, and I can cover it by myself in general if we will find someone.
Sounds good, but let's split the costs, there are probably others that would like to contribute as well.
@sTix, George Seems that we found a developer, so once it will be closed to pay off, I will write there. As i say i will pay from my own, but if some of you will cover a little bit that will be good as well :)
There are plenty of bounty websites out there that we can use to gather the amount of money. What do you think?
@kas1e Do you think it would be good to put the information on how to use gprof on wiki.amigaos.net as well? Maybe, if you agree, I could include it in the new SDK I am working on.
@walkero There is nothing too big in this deal to gather money and worry about bounties :) I just will pay on my own when (And if) the time comes. We have enough in the past collections and bounties :)
As for the wiki, sdk, and so on: of course, but not now, as for now, all that gprof stuff didn't work as should work, need fixing, testing, and stuff.
Seems that we found a developer, so once it will be closed to pay off, I will write there. As i say i will pay from my own, but if some of you will cover a little bit that will be good as well :)
Sorry, tl;dr, but once there was a very well working gprof version for AmigaOS 4.x. Don't remember the details why it couldn't be updated any more, but IIRC it was libgcc related. Maybe it even still worked on some cross compilers, but I always refused to use them. If even something as "simple", or rather portable, like GCC and it's tools can't be made to work on AmigaOS, why still use AmigaOS? At least a 99% working compiler, incl. it's tools, is the minimum requirement for any serious development. Most users of course don't understand the extreme consequences of unusable or missing development tools, but expect to get something like FireFox or Chromium to get ported to AmigaOS, which is nearly impossible without the development and bug fixing tools ...
Absolute, if the OS did not crash 80% of the time when I run configure, that be real game changer, having to re do the build system, before even starting working on something is just huge wall to climb over. I can wave white flag, use cross compiler, but that’s not what I like to do.
"Grep" is one of thing on my mind, reclusive grepping a directory be really nice.
"Awk" crashes a lot.
(NutsAboutAmiga)
Basilisk II for AmigaOS4 AmigaInputAnywhere Excalibur and other tools and apps.
Absolute, if the OS did not crash 80% of the time when I run configure, that be real game changer
Unless you get reproducible crashes from a single tool like awb or grep try using a few MB more stack, stack overflows were the usual reason I got random crashes with configure or gmake. IIRC I used a 8-10 MB stack for building OWB and 4-5 MB for anything else.
As for making grof work again: Try building your software as REL executables (like it was done in the early versions of AmigaOS 4.0 and ADTools) instead of EXEC ones, that might still work. At least for the CPUs supported back then (603, 604, 750, 64xx, etc.), for POWER CPUs (X1000) and newer PPC ones (X5000, X1200, sam460) CPU specific profiler code in the complier and/or the C libraries is required if nobody added it yet.
Edited by joerg on 2022/2/4 18:49:37 Edited by joerg on 2022/2/4 18:51:08 Edited by joerg on 2022/2/4 18:53:07 Edited by joerg on 2022/2/4 19:21:49 Edited by joerg on 2022/2/4 19:23:12
We found out why it didn't work and found out how to make it works (that was the last message on that topic on the last page of the topic before your answer).
In brief, the issue is that grpof is working as it works before, all fine. Even the latest version has no problems. The issue is with the binaries which since some time (when sobjs was introduced) have code segment started now from 0x01000000, while before, in 2008, it was 0x00000000. While it is just a placeholder, the profiling code in the clib2 and newlib was built with hardcore 0x00000000. So, since time when things changed in the ldscripts to have by default placeholder as of 0x01000000, profiler's code in clib2 and newlib didn't work correctly anymore.
To make it works, one needs to add "-Ttext=0x00000000" , and that all.
Through that is just a workaround, because it all should work by default, meaning, that profiling code in clib2 and newlib need to be a bit fixed and that is all.
But that is just half of the problem. Another moment is that grpof (more the profiling code in clib2 and newlib), use "performance monitor" resource, which wasn't presented anywhere, but only in pegasos2, old a1 and old classic machines, and only a few weeks ago, Mathias (Corto) are added it to X5000 and Tabor, and for now, trying to fill latest bits here and there.
After performance monitor on x5000/tabor will work the same as on all other older platforms, and x1000 support will be in as well, then clib2 and newlib profilers can be fixed to not have hardcoded 0x0000000.
But that is just a half of problem. Another moment, is that grpof (more the profiling code in clib2 and newlib), use "perfomance monitor" resource, which wasn't presented anywhere , but only in pegasos2 , old a1 and old classic machines, and only few weeks ago , Mathias (Corto) are added it to X5000 and Tabor, and for now, trying to fill latest bits.
Profiling needs CPU specific code, back then only 603, 604, 750 and 64xx CPUs (classic Amigas, A1XE/Pegasos, sam440) were supported. POWER CPUs (X1000) and newer PPC ones (X5000, X1220, sam460) would require updated CPU specific parts. Ideally it should be done in the kernel performance monitor, but if nobody does it there it could be added to the C libraries instead.
If something doesn't work at all with GCC and newlib/clib2 don't forget there is still VBCC with it's vclib as an alternative. Not really usable for porting software, but for AmigaOS native software no big difference, and in case something is missing or doesn't work much easier to add or fix.
News feed
OS4 Feedback forum
OS4Depot Feedback forum
Amigabounty forum
OpenAmiga forum
AmiCygnix forum
ABC shell forum
AmiKit forum
Cinnamon Writer forum
CodeBench forum
Digital Universe forum
Dopus 5 forum
E-UAE forum
Gnash forum
Ibrowse forum
JAmiga forum
Odyssey forum
OWB forum
Qt forum
SmartFileSystem forum
Timberwolf forum
TouchDevice forum
TuneNet forum
Unsatisfactory Software forum
Donate
