Develop (with) shared objects

Hi,

I tried to compile software with shared objects (a new feature available from the last SDK) and to create one. I have many questions and as the topic was almost never discussed, I would like to group information here. And I also plan to write an article on gurumed.net

To start, I did read the nice Frieden's article on shared objects.


1. What are the difference between -rpath et -rpath-link ? What do they do exactly ?

2. Is it planned to add the version tag in shared objects provided in the system ? It would be easier for updates and version control.

3. Should we check the availability of shared objects in SOBJS: at the beginning of our programs ? Does it crash if the shared object is missing ? I did not tried yet.

4. To avoid links in the SDK lib directory, would it be possible for the compiler to check the path SOBJS: first ? In the article, it is told that "-lSOBJS:" can be added in order to find shared objects in SOBJS:, is it really the case ? It does not seem to work here.

5. For the provided shared objects, could you also put the includes in the SDK ? I think about freetype, I am confused with 2 old libs from os4depot and a shared object without includes (maybe I overwrote them ...).

6. Could someone document more about shared objects, options (like syntax about -rpath, etc.) ?

7. I wonder how versions will be managed because we have already started to create links concerning libSDL / libSDL-1.2, libpng / libpng12, ...

@corto

2.

Perhaps you could ask Rigo to speed this project up? ;)

http://openamiga.org/index.php?function=viewproject&projectid=32

@corto

Quote:
http://en.wikipedia.org/wiki/Rpath_%28linking%29

Quote:
yes. is planned

Quote:
The new elf.library will tell you if a (and which) shared object is missing and will print a "file is not executable" when you click on ok

Quote:
IIRC it will first check the PROGDIR: and then SOBJS: (or the countrary). Snoopy will tell you
I prefer to put my .so in SOBJS: because actually there are a lot of differents .so files. we need a good repository

Quote:
The include files are same of the staic version one

Quote:
I think google will help you alot.. :D

Quote:
Usually .so files came with a version number, so you can have multiple version libraries and usually the one without number is a link to the last version.

@corto

Quote:

that should be a capital 'L' ie

-LSOBJS:

also it may be that it should be

-L/SOBJS

as gcc uses unix paths

this only effects compile time IIRC, at runtime SOBJS: is autmatically searched, whether included at compiletime or not.

@broadblues : Yep, it is of course -L, it was a typo. I tried with -LSOBJS: and the object is not found. Only -L/SOBJS seems to work.

@orgin : Thanks for the link !

@afxgroup

Thanks for all your answers.

Quote:

Ok. Just checked and it does not crashed. Nice.

Quote:

With the option "-Wl,--verbose", gcc look into the SDK paths before the ones given with -L. For example my libSDL_ttf.so in the current directory is not chosen because libSDL_ttf.a is found before.

Quote:

My problem is that it seems that the good includes are in "SDK:Local/common/include/freetype2/freetype/"

Quote:[/quote]

Ok. Understood.

So except the strange freetype2 inclue directory, it shoud be good.

@corto

Quote:

Why do you consider it strange? In my sdk most include are in sdk:local/common

having said that the slight difference in path between the lib and include directories, does seem to make calling ./configure harder. I always seem to need to fix config.status by hand.

@broadblues

The strange thing is I can't remember if some freetype includes were in the newly installed SDK. The other day, I needed freetype and installed libs and includes from os4depot but there were 2 archives there.

Now, in "SDK:Local/common/include", I have :
- a directory "freetype" with a version 1.1.x or 1.2.x inside
- a directory "freetype2/freetype" with a version 1.3.5 inside if I remember correctly, I don't understand why there is this directory freetype2 ...

Even if freetype includes were in the SDK, I don't think static libs were provided else I would not have added these libs.

If I compile my shared SDL_tff with includes in "freetype2/freetype" and I use this shared object, it runs but with includes in "freetype", the final executable crashes.

@corto

Hmmm I have both sets of includes too.

I have a libfreetype.so dates 2008 and a libfreetype.a dated 2006.

I'm totally unsure as to which version I'm using in blender, but if I enable freetype fints I get acrash on exit, I wonder if I have a similar mismatch to you? Have to put that on my things to check list.

@broadblues

you will use .so only if you add -use-dynld otherwise the linker will use .a
BTW it is sure that if you mix .so that are linked statically with .a, and same .a library in .so format you will get a crash for sure.
The correct way is create shared libraries that are linked with other .so libraries instead of .a

@afxgroup

Quote:

I'm using the .a I know this for certain. (it's explictly referenced on the linker command line) What I wasn't sure about is which includes I was using. Haven't had time to check yet.

Quote:

Sorry I didn't get waht mean there, can you rephrase?

@afxgroup

Quote:

Right but .so will be used only if .a is not found before on the path.

About freetype includes, I think the most clean solution would be to provide the .so file, its includes (not in freetype2/) and also the static libs of the same version.

@broadblues

eheh.. i was posting an example.. but i'm usually a lazy man..

So,

if you have
libtest.so linked with libtest2.a
libtest2.so

and then you use in the same program libtest.so and libtest2.so (the .so version of libtest2.a) you will have a lot of problems..

yes. it is a bit weird..

@corto

Quote:

IIRC if you use dynld it will use always .so file *except* if you specify the library with -L, but if you use -ltest it will find first for libtest.so and then libtest.a

I've just tried to summarize all the points that were unclear and then discussed here. Please read and comment, I would like to put all information in one place later :


-use-dynld

It allows to use shared object that will be loaded / linked at runtime. But at compilation time, note that for each path it will check first if a shared object exists, else it will use the static library.
So you might think you will use a shared object because you specified -use-dynld but it could not be the case.


-Lpath

This classic option tells to the linker where it has to search the objects.
It will look first in the SDK paths.

[source]attempt to open /SDK/newlib/lib/libfreetype.so failed
attempt to open /SDK/newlib/lib/libfreetype.a failed
attempt to open /GCC/lib/gcc/ppc-amigaos/4.2.4/newlib/lib/libfreetype.so failed
attempt to open /GCC/lib/gcc/ppc-amigaos/4.2.4/newlib/lib/libfreetype.a failed
attempt to open /SDK/local/newlib/lib/libfreetype.so failed
attempt to open /SDK/local/newlib/lib/libfreetype.a failed
attempt to open /SDK/local/common/lib/libfreetype.so failed
attempt to open /SDK/local/common/lib/libfreetype.a failed
attempt to open ./libfreetype.so failed
attempt to open ./libfreetype.a failed
attempt to open /SOBJS//libfreetype.so succeeded
-lfreetype (/SOBJS//libfreetype.so)[/source]


-lW,-rpath,<runtimepath>

This is a linker option and as indicated, it contains a runtime path (see : http://en.wikipedia.org/wiki/Rpath_%28linking%29) that will be hardcoded in the final ELF executable. It is a bad idea to use a SDK related path ! This path "may either override or supplement the system default dynamic linking search path" even if I suppose that the object is search into SOBJS: first on OS4.


Check which shared objects are used

If you compiled "myprogram" and you want to see which dependencies are shared objects, do :
readelf -d myprogram

Here is an example of results, you will see used shared objects looking at lines which type is "(NEEDED)" :

[source]> readelf -d example

Dynamic section at offset 0x5d6c contains 19 entries:
Tag Type Name/Value
0x00000001 (NEEDED) Shared library: [libSDL-1.2.so]
0x00000001 (NEEDED) Shared library: [libfreetype.so]
0x00000001 (NEEDED) Shared library: [libz.so]
0x00000001 (NEEDED) Shared library: [libgcc.so]
0x00000001 (NEEDED) Shared library: [libc.so]
0x00000004 (HASH) 0x10000e8
0x00000005 (STRTAB) 0x100051c
0x00000006 (SYMTAB) 0x100023c
0x0000000a (STRSZ) 617 (bytes)
0x0000000b (SYMENT) 16 (bytes)
0x00000015 (DEBUG) 0x0
0x00000003 (PLTGOT) 0x1015e6c
0x00000002 (PLTRELSZ) 408 (bytes)
0x00000014 (PLTREL) RELA
0x00000017 (JMPREL) 0x10007a0
0x00000007 (RELA) 0x1000788
0x00000008 (RELASZ) 432 (bytes)
0x00000009 (RELAENT) 12 (bytes)
0x00000000 (NULL) 0x0[/source]


Dependencies

If a shared object it used, its own dependencies must be shared objects too. For example, I tried to test the "libSDL_ttf.so" file I compiled but as the "libfreetype.a" in the SDK subdirectories was found before the "libfreetype.so" in SOBJS: and that can't work ! In that case, prefer the static library (SDL_ttf is small) and create link of the shared object in the path "SDK:Local/newlib/lib" so it will be found at compilation time and "libfreetype.so" will be used, as you specified -use-dynld.


At runtime

We saw that shared objects are stored in SOBJS: (what means SYS:SObjs/) and the ELF loader (elf.library) will look into the path specific with the option -rpath if used. If an object is not found, a window will appear to tell which object is missing and at the end the message "file not an executable" will be printed in a console.


Version number

With the provided shared objects, the command "version" will give no results because the version was not added. Some shared objects have a version in their name (SDL-1.2, png12, ...) and I was wondering how they are managed knowing that these libraries also exist with their plain name (SDL, png, ...). Usually .so files came with a version number, so you can have multiple version libraries and usually the one without number is a link to the last version.

I reply to myself because I think I've just found a very important information to share.

My example of shared objet libSDL_ttf.so was crashing with an ISI error. I used to compile it like that :

gcc -shared -o libSDL_ttf.so SDL_ttf.o

I followed the procedure that I found but in my case, objects to link are also shared objects so it must be :

gcc -L/SOBJS/ -use-dynld -shared -o libSDL_ttf.so SDL_ttf.o -lfreetype

Tell me if I am wrong but now that works ! We have to explicit the fact that creating the shared object, it has to be linked dynamically with another one, this is why I added -use-dynld and -lfreetype.

I hope all that will help !

@corto

You don't need -use-dynld if compiling a shared object with -shared it's implied bt the -shared option, but yes you are right to add any dependent libraries to the link line.


Another thing to note:

If you are using shared objects as plugins rather than as libraries, then they may require to link against the main executable. In thie case you need to add -W,--export-dynamic to the main executable link command. (or just --export-dynamic if you are calling ld directly)