Modestas Vainius
2011-Mar-14 22:23 UTC
Handling BIC-without-SONAME-bump in KDE SC libraries
Hello, I personally can''t comment on how much upstream improved after the last discussion (iirc, on kde-core-devel). However, it would be naive to think that we will never face BIC-without-SONAME-bump problems in the future. Solving them all upstream would be great but it''s simply too optimistic scenario due to many factors. Generally I can think of two ways of dealing with this. The first is our policy at the moment. 1. [ Package level ]. Rename the library package and add Breaks/Replaces against the previous versions, keep library SONAME (and filenames) unchanged. Pros: a) binary compatible with upstream and other distributions (at least in theory); b) ensures users have no old BIC libs (and apps linked to them) around. If an app ends up directly or indirectly linked to the old and new library with the same symbols, bad things may happen (e.g. recent libpng case and see below); c) does not require patching of upstream source. Cons: a) apt unfriendly because two library packages cannot be co-installed. Library packages are typically deep in the dependency chain so apt may require to force removal/installation in some cases; b) makes partial upgrade to new KDE releases very painful. I.e. it renders some 3rd party applications uninstallable due to conflicting library packages; 2. [ Library level ]. Change library SONAME (e.g. add debN suffix, specifics to be discussed) and rename the package. No Breaks/Replaces needed as there are no conflicting files. Pros: a) apt friendly as it''s a recommended way to deal with SONAME changes; b) new KDE releases should no longer break 3rd party applications (at least at the package management level); Cons: a) custom SONAMEs and library filenames will be binary incompatible with the rest of the world; b) if conflicting libraries are loaded in the same app memory space (unlikely though), it might lead to crashes at runtime; c) crashes might occur due incompatibilities in the runtime client/server protocols (if there are ones); d) requires patching of upstream source (i.e. altering SOVERSION in the CMakeLists.txt); So what I am proposing is the switch to 2). I do believe that user experience is more important than SONAME compatibility with the rest of the world (which is theoretical anyway, at least in the KDE land). IMHO, other cons are not a big deal either as KDE is not libpng and runtime conflicts are more unlikely than likely. So what do you think and which way do you prefer? Maybe we could think of something better? -- Modestas Vainius <modax at debian.org> -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: <http://lists.alioth.debian.org/pipermail/pkg-kde-talk/attachments/20110315/da2c08de/attachment.pgp>
Modestas Vainius
2011-Mar-15 22:44 UTC
Handling BIC-without-SONAME-bump in KDE SC libraries
Hello, On antradienis 15 Kovas 2011 00:23:14 Modestas Vainius wrote:> 2. [ Library level ]. Change library SONAME (e.g. add debN suffix, > specifics to be discussed) and rename the package. No Breaks/Replaces > needed as there are no conflicting files.As an amendment to this plan, we could start versioning the symbols of BIC- prone libraries (i.e. basically everything not kde*libs). This would help us avoid the following con:> b) if conflicting libraries are loaded in the same app memory space > (unlikely though), it might lead to crashes at runtime;It''s still an open question how much symbol versioning would affect people building from source (e.g. kde developers or kdesrc-build users). What''s more, it''s still unknown how much effort (i.e. upstream code patching) this would need. If we were to move forward with this plan, we need to agree about naming of custom SOVERSION and symbol versions. Please respond to this mail even if it was a short yes/no answer with small remarks. My proposal would be: SOVERSION: ${upsteamSOVERSION}debN (where N is a number (starting from 1) bumped after each BIC-without-SONAME-bump) symbol versions: DEB_${upsteamSOVERSION}[_N] where N is the same as above if there was at least one BIC-without-SONAME-bump. Otherwise _N part would be omitted. Whereas symbol versioning could be avoided if there was no BIC- without-SONAME-bump, is an open technical question (that would be somewhat more binary compatible with the rest of world, at least temporary). -- Modestas Vainius <modax at debian.org> -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: <http://lists.alioth.debian.org/pipermail/pkg-kde-talk/attachments/20110316/d0a1a757/attachment.pgp>
Lisandro Damián Nicanor Pérez Meyer
2011-Mar-16 01:51 UTC
Handling BIC-without-SONAME-bump in KDE SC libraries
On Tuesday 15 March 2011 19:44:28 Modestas Vainius wrote:> Hello, > > On antradienis 15 Kovas 2011 00:23:14 Modestas Vainius wrote: > > 2. [ Library level ]. Change library SONAME (e.g. add debN suffix, > > specifics to be discussed) and rename the package. No Breaks/Replaces > > needed as there are no conflicting files. > > As an amendment to this plan, we could start versioning the symbols of BIC- > prone libraries (i.e. basically everything not kde*libs). This would help > us > > avoid the following con: > > b) if conflicting libraries are loaded in the same app memory space > > (unlikely though), it might lead to crashes at runtime; > > It''s still an open question how much symbol versioning would affect people > building from source (e.g. kde developers or kdesrc-build users). What''s > more, it''s still unknown how much effort (i.e. upstream code patching) > this would need. > > If we were to move forward with this plan, we need to agree about naming of > custom SOVERSION and symbol versions. Please respond to this mail even if > it was a short yes/no answer with small remarks. My proposal would be: > > SOVERSION: ${upsteamSOVERSION}debN (where N is a number (starting from 1) > bumped after each BIC-without-SONAME-bump) > > symbol versions: DEB_${upsteamSOVERSION}[_N] where N is the same as above > if there was at least one BIC-without-SONAME-bump. Otherwise _N part would > be omitted. Whereas symbol versioning could be avoided if there was no > BIC- without-SONAME-bump, is an open technical question (that would be > somewhat more binary compatible with the rest of world, at least > temporary).As for me, sounds good. I also want to offer myself to put this on a webpage/wiki once it gets completely decided (or even before it''s needed somehow). Regards, Lisandro. -- Hiroshima ''45, Chernobyl ''86, Windows ''95. Grafitti en Niceto Vega 5940, Buenos Aires. De una foto de Mario Gallo. Lisandro Dami?n Nicanor P?rez Meyer http://perezmeyer.com.ar/ http://perezmeyer.blogspot.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part. URL: <http://lists.alioth.debian.org/pipermail/pkg-kde-talk/attachments/20110315/6027bb6d/attachment.pgp>
Modestas Vainius
2011-Mar-25 10:25 UTC
Handling BIC-without-SONAME-bump in KDE SC libraries
Hello, as SONAME changes + symbol versioning seems to be the most viable option, I did some experiments on what end effect it might have and how all this would look like. Glossary -------- unstable-ABI library - a library which is prone to break BC without SONAME change. Undefined (UNDEF) symbol - a plain symbol reference. It will be resolved to the implementation/definition in the external shared library by dynamic linker. Conclusions ----------- 1) ABI-unstable libraries should get all symbols versioned from the start. That''s because undefined unversioned symbols may bind to any symbol of the same name in any library which dynamic linker (recursively) loaded [1]: | In case only the object file with the reference does not use | versioning but the object with the definition does, then the reference | only matches the base definition. [ ...] If there is no symbol definition | with such an version index and there is exactly one version for which this | symbol is defined, then this version is accepted It does not matter if a real symbol in the library is versioned or not. The first same-name real symbol seen by dynamic linker will be bound to the unversioned UNDEF. It means that we basically have no control over this unless we version the symbols. 2) Unlike unversioned (Base) symbols, versioned UNDEF symbols have version *and* originating library information assigned to them. According to [1]: | The last case is if the object with the references uses symbol | versions but the object with the definitions has none. In this case a | matching symbol is accepted unless the object''s name matches the one | in the Elfxx_Verneed entry. In the latter case this is a fatal error. However, I determined that this is not the case since middle 1999 [2]. This situation is not a fatal error but merely a warning [3]. However, I''m pretty sure that it''s a bug in the dynamic linker which may get fixed one day. Respective assert() is in place [4] but it does not fire in release builds. 3) Versioned symbols cannot be easily "moved" from one library to another down the library dependency tree without breakage (as it was done e.g. with "libkutils4 -> libkcmutils4 libkemoticons4 libkidletime4 libkprintutils4" split). But this should be a non-issue for unstable-ABI libraries. Having in mind the points 1 and 2 above, all symbols in the unstable-ABI libraries should get a unique-per-ABI version. As soon as upstream breaks BC without bumping SOVERSION, we bump SONAME and change a package name. However, initially, we keep original upstream SONAME. This strategy allows to keep some compatibility with the rest of the world as long as upstream does not break BC without bumping SOVERSION so we do not have to bump SONAME ourselves. In particular: a) external binaries (which supposedly have unversioned UNDEF symbols as pushed by upstream, see point 1) will work with unstable-ABI libraries as packaged by Debian; b) binaries with versioned UNDEF symbols (i.e. linked against our unstable- ABI libraries) will work with unstable-ABI libraries as built from upstream sources (kdesrc-build) or packaged by other distros which do not use incompatible symbol versioning. That''s because of point 2 and as long as the linker bug is not fixed. So how to handle BIC? --------------------- As said in previous mails, we need to choose a custom string for our SONAMEs (that is either SOVERSION or OUTPUT_NAME suffix, whatever toolchain eats better) and symbol versions. Let it be "abiN" (where N is an integer) for SONAME bumps and ABI_N for symbol versions respectively (with _0 for unchanged SOVERSION). So in 4.6, we add symbol versions ABI_0 to all unstable-ABI libraries which kept BC throughout 4.4 -> 4.6 cycle. Otherwise, if libs broke BC, we rename their packages, bump their SONAMEs by adding "abi1" suffix and set their symbol versions to ABI_1. As 4.6 is going to be a transitional release from unversioned to versioned symbols, we might consider adding "Breaks" against old libs (<< 4:4.6) to all future "abi1" packages. That''s to avoid weird crashes if both old unversioned and new versioned BIC libraries end up getting loaded in the same process space. While it''s possible, it''s also highly unlikely because unstable-ABI libs are not very popular (unlike e.g. libjpeg). So let''s not do this "Breaks" stuff at first and see what happens. Implementation details ---------------------- In order to minimize patching, I propose the following scheme. All unstable- ABI library packages would be marked with a custom "X-Custom-ABI: N" (or something like that) debian/control field. Then we would need to implement a cmake script which would parse debian/control, bump SONAME and add appropriate -Wl,--version-script,script-name to the linker command line on- the-fly. This script would be shipped in pkg-kde-tools to be INCLUDEd from the bottom of /CMakeLists.txt of the each source package that needs this capability. That would effectively limit us to one patch per source package while the rest would be conveniently "configured" in debian/control. [1] http://www.akkadia.org/drepper/symbol-versioning [2] http://sourceware.org/git/?p=glibc.git;a=commit;h=9a8fcca0b33c26759134a545ac45251df53418a3 [3] <apppath>: <libpath>: no version information available (required by <apppath>) [4] http://sourceware.org/git/?p=glibc.git;a=blob;f=elf/dl-lookup.c;hb=glibc-2.13#l168 -- Modestas Vainius <modax at debian.org> -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: <http://lists.alioth.debian.org/pipermail/pkg-kde-talk/attachments/20110325/1e6f852b/attachment.pgp>