libjpeg, and other shared object versioning phenomenon

I've just spotted (no pun intended) Spot's libjpeg.so 6b on OS4Depot, and it made me do some digging.

libjpeg 6 - as Spot points out - is the version everything is compiled against.
libjpeg 7 appears to be the version included with OS4.1u2 (this doesn't work with libjpeg 6 compiled apps)
libjpeg 8 is the latest (doesn't work with libjpeg 6 compiled apps, probably doesn't work with libjpeg 7 compiled apps either)

I've just recompiled libjpeg 8b so it has a soname of libjpeg.so.8. This half resolves the problem of incompatibility - apps wanting to use the newer version can, without upsetting the older apps.

The big issue comes when a shared object references libjpeg, and an app wants to use this shared object as well as libjpeg. All the libjpeg references must be for the same version or it won't work. If you have a shared object linked to libjpeg 6, if you try and link your app to this shared object and libjpeg 8, it won't work as expected.

This is resolveable by ensuring the app and the shared object are linked to the same version of libjpeg. Sounds easy enough, until you try and compile something which requires the newer version, which the shared object isn't compatible with. If you relink the shared object to the new version, this could break other software. You could give it a new name, but that's creating unnecessary clutter.

This problem is also present with libcairo and libpng - it is impossible to link an app which needs both libpng 1.4 and libcairo, as libcairo is linked against libpng1.2.

Am I missing something obvious here, or are shared object version dependencies deeply flawed? I don't think this is an OS4 issue either - how does Linux avoid this particular problem?

@Chris

I did the libjpeg v8a build. I had tested it, but perhaps not well enough. I personally hadn't seen any problems and all the references appeared to be the same.

That isn't so with libpng (i've never released any shared libpng, but have compiled it here), and of course when compiling libpng, the configure script makes sure the shared object has a name reflecting the version of libpng, not just libpng.so, although a symlink is created I think (Edit: Though as I say in the last paragraph of this post, that's OK because any program executable won't be loading sobjs:libpng.so)

I don't like renaming things. If an updated version doesn't change the name of the shared library, then I won't either.

I did notice spotUP's build of libjpeg-6b.so was renamed internally with -soname i'd presume, however naming it and copying it as libjpeg.so won't work properly. All old and new programs will find sobjs:libjpeg.so, which is 6b, and want to use it, however libjpeg.so references libjpeg-6b.so which will not exist.

From experience, if you have sobjs:libjpeg.so which is named internally as libjpeg-6b.so and you link your program with -ljpeg, then in the executable of your program, it will look for sobjs:libjpeg-6b.so

@MickJT

thanx for spotting that!
i wanted my lib to be named libpng-6b.so at first, but then i changed my mind.
and obviously i forgot the internal name.
the archive is deleted.
i will upload a fixed one today.

@spotUP

fixed dev and user archive uploaded on os4depot.

@spotUP
Quote:
I noticed there is nothing about soname mentioned in the current SDK documentation on shared objects. Could you please file a bug report against the SDK on this?

@Chris

Thanks, this summary will help. I didn't make it as clear.

I have to do some tests. Yesterday I added traces in SDL_image, I would like to understand why that doesn't work with this version of libjpeg.

@corto

Maybe not supported? Try libjpeg 8. Use my renamed .so version which (edit 09/2010) I have uploaded here

@SpotUP

It may have been better to keep the name, rename the file back to what it was supposed to be, and create a softlink from libjpeg.so to libjpeg-6b.so. That way anything in the future will be linked directly to 6b and we avoid this problem. But, tbh, it's probably a bit late for that now - libjpeg.so should have had a version number in from the start, but I guess nobody realised major versions were incompatible with each other.

Quote:

Thanks! I haven't tried it yet but I have to say that I became crazy about some incompatibilities found with libjpeg and SDL_image ...

Quote:

I realised and said that when I tried to port Pygame and I had problems with PointRider (that uses SDL_image), what caused a delay ...

I've found the reason why libjpeg versions are not compatible : in the decompression function, there is a version check. This is why if libjpeg 6.2 is replaced by version 8 in the system, all programs that were using it will fail.
As the version 8 doesn't bring any major enchancements, I would prefer to stick with version 6.2. I think the dev team should have the control on the essential shared objects, so every developers will have to be careful before releasing a new version.

@corto

Quote:

That's all very well, but libjpeg wasn't included with OS4 until one of the updates - prior to that, v6 was available for download and being used. The update brought v7 under the same filename.

Also, what happens if a program does require a newer version of an included SObj? You can't restrict the building of open source SObjs to the OS4 devs; I'd argue they have better things to do anyway.

@Chris

A libjpeg with a version different from the one used to build SDL_image returns an error. The following behaviour depends on how the program handles it.

Else, I've also seen some problems at compilation time, I don't remember if it was with libjpeg though.

Right, we can't restrict anyone to do anything :)
But for heavily used shared objects, it can be dangerous and can make things difficult to keep a clean environment, with shared objects and static libs in the SDK, shared objects in SOBJS:, links to make things matching, ...

For me, with what I've experienced, I think I will release PointRider with static libs even if it has a big size (several MB).

@Chris

Actually the fastest solution is not to install the sobjs into SOBSJ: folder until we find a good solution but to create a SIBJS/ dir into your PROGDIR: this will avoid any problem. And of course we should start to add a version to the name to avoid those problems

@afxgroup

I hope there will be a naming template, I mean there are several possibilities :
- libjpeg-6.2.so
- libjpeg62.so
- libjpeg.so.6.2

I prefer the first one and I think it is commonly used but who knows ...

Anyway, what is the interest in putting shared objects in the program directory compared to link statically ?

@corto

we should use the third since it is the "official" linux naming and a lot of scripts use that naming convention

Quote:
Typically by recompiling programs that are linked against the old library so that they after the recompile are linked against the new one. Then you remove the old one. That's how I prefer to do it at least.

@afxgroup

I thought the usual naming technique was libjpeg.so.6

Only the major revision number. So 6.1, 6.2, 6.3 all use libjpeg.so.6 with a symlink to libjpeg.so and libjpeg.so.6.2 etc..

Edit: Btw, thread title should say "phenomena" not "phonemenon" </pedant>

@MickJT

iirc on linux all real lib has .so extensions while all other versions (.6.2 and so on) are symlink. but maybe i'm wrong... we need someone that use linux to confirm this