It did partially fix the issue but there is still one problem: llvm-objdump -sx gives: Version References: required from libpthread.so.0: 0x09691a75 0x00 05 GLIBC_2.2.5 0x09691972 0x00 09 GLIBC_2.3.2 0x09691973 0x00 07 GLIBC_2.3.3 0x06969192 0x00 12 GLIBC_2.12 required from libdl.so.2: 0x09691a75 0x00 21 GLIBC_2.2.5 required from libuuid.so.1: 0x09da27b0 0x00 19 UUID_1.0 required from libm.so.6: 0x09691a75 0x00 04 GLIBC_2.2.5 0x06969187 0x00 06 GLIBC_2.27 0x06969189 0x00 11 GLIBC_2.29 required from libc.so.6: 0x09691a75 0x00 03 GLIBC_2.2.5 0x0d696913 0x00 14 GLIBC_2.3 0x09691972 0x00 08 GLIBC_2.3.2 0x09691974 0x00 17 GLIBC_2.3.4 0x0d696914 0x00 16 GLIBC_2.4 0x0d696917 0x00 13 GLIBC_2.7 0x06969191 0x00 18 GLIBC_2.11 0x06969194 0x00 15 GLIBC_2.14 0x06969197 0x00 10 GLIBC_2.17 required from ld-linux-x86-64.so.2: 0x0d696913 0x00 20 GLIBC_2.3 So there are still some references to GLIBC_2.29 from libm. It is in the VERSION_NEED or something similar I think. Could llvm-strip figure that it can get rid of it? Regards, Alexandre Bique On Wed, Nov 11, 2020 at 2:25 PM Alexandre Bique <bique.alexandre at gmail.com> wrote:> > This did the job: > > patchelf --clear-symbol-version log Podolski.64.so > > Alexandre Bique > > On Wed, Nov 11, 2020 at 2:19 PM Alexandre Bique > <bique.alexandre at gmail.com> wrote: > > > > On Tue, Nov 10, 2020 at 7:01 PM Fāng-ruì Sòng <maskray at google.com> wrote: > > > > How to find what is pulling libmvec? > > > > > > If you build a -DCMAKE_BUILD_TYPE=Debug lld, you can set a breakpoint > > > on lld/ELF/MarkLive.cpp:114 (ss->getFile().isNeeded) and check which > > > symbol makes > > > libmvec.so needed. > > > > I'm afraid this is going to far for me :) > > > > > > On the other hand, I don't understand why I have to specify that I > > > > want exp from glibc 2.2 or 2.29, I just want the exp symbol, and if > > > > possible the latest version available on the system. > > > > Because I don't have a high precision requirement so they would all be > > > > fine. Is that possible? > > > > > > Symbol versioning is complex ;-) While the runtime behavior is > > > documented by Linux Standard Base Core Specification, Generic Part, > > > there is no good documentation on the linker behavior. > > > LLD has evolved to the current state after contributors "observe GNU > > > ld behaviors and patch LLD when needed" :( > > > > I think there might be some discussions needed with > > glibc/coreutils/gcc/llvm because it is way too complicated. > > I believe that targeting an older glibc version should not be more > > complicated than > > -DGLIBC_COMPAT_LEVEL=0x020205 -Wl,compat=GLIBC_2.2.5 > > > > On OSX there is a target platform command line option, and I believe > > it is the right approach, yet I can't really comment on it. > > > > > For an undefined symbol, there is only this form: sym at ver > > > For a defined symbol, both sym at ver (non-default; sometimes called > > > hidden) and sym@@ver (default) exist. > > > > > > For a defined symbol named sym at ver, it only resolves an undefined > > > symbol sym at ver. > > > For a defined symbol named sym@@ver, it resolves both undefined > > > sym at ver and undefined sym. > > > > > > If you have glibc 2.29, there are exp at GLIBC_2.2.5 (non-default) and > > > exp@@GLIBC_2.29 (default), your 'exp' undefined symbol > > > will resolve to exp@@GLIBC_2.29 (default) and not work with older glibc. > > > > > > If you know that your translation unit does not define 'exp' (usual > > > case), you can write > > > > > > .symver exp,exp at GLIBC_2.2.5 > > > call exp > > > or > > > .symver exp,exp@@@GLIBC_2.2.5 > > > call exp > > > > > > Then the undefined symbol 'exp' will be renamed to 'exp at GLIBC_2.2.5' > > > and get bound to the old version at link time. It will thus work with > > > old glibc. > > > > I have one .cpp file with: > > > > __asm__(".symver powf,powf at GLIBC_2.2.5"); > > __asm__(".symver expf,expf at GLIBC_2.2.5"); > > __asm__(".symver exp2f,exp2f at GLIBC_2.2.5"); > > __asm__(".symver log2f,log2f at GLIBC_2.2.5"); > > __asm__(".symver logf,logf at GLIBC_2.2.5"); > > > > __asm__(".symver log,log at GLIBC_2.2.5"); > > __asm__(".symver log2,log2 at GLIBC_2.2.5"); > > __asm__(".symver exp,exp at GLIBC_2.2.5"); > > __asm__(".symver exp2,exp2 at GLIBC_2.2.5"); > > __asm__(".symver pow,pow at GLIBC_2.2.5"); > > > > But it does not solve the issue. I've tried many variations. > > > > Do you know if it is possible after the link, to edit the shared > > library and replace the dependency on GLIBC 2.29 to GLIBC 2.2.5? > > > > Regards, > > Alexandre
On 2020-11-11, Alexandre Bique wrote:>It did partially fix the issue but there is still one problem: > >llvm-objdump -sx gives: > >Version References: > required from libpthread.so.0: > 0x09691a75 0x00 05 GLIBC_2.2.5 > 0x09691972 0x00 09 GLIBC_2.3.2 > 0x09691973 0x00 07 GLIBC_2.3.3 > 0x06969192 0x00 12 GLIBC_2.12 > required from libdl.so.2: > 0x09691a75 0x00 21 GLIBC_2.2.5 > required from libuuid.so.1: > 0x09da27b0 0x00 19 UUID_1.0 > required from libm.so.6: > 0x09691a75 0x00 04 GLIBC_2.2.5 > 0x06969187 0x00 06 GLIBC_2.27 > 0x06969189 0x00 11 GLIBC_2.29 > required from libc.so.6: > 0x09691a75 0x00 03 GLIBC_2.2.5 > 0x0d696913 0x00 14 GLIBC_2.3 > 0x09691972 0x00 08 GLIBC_2.3.2 > 0x09691974 0x00 17 GLIBC_2.3.4 > 0x0d696914 0x00 16 GLIBC_2.4 > 0x0d696917 0x00 13 GLIBC_2.7 > 0x06969191 0x00 18 GLIBC_2.11 > 0x06969194 0x00 15 GLIBC_2.14 > 0x06969197 0x00 10 GLIBC_2.17 > required from ld-linux-x86-64.so.2: > 0x0d696913 0x00 20 GLIBC_2.3 > >So there are still some references to GLIBC_2.29 from libm. >It is in the VERSION_NEED or something similar I think. > >Could llvm-strip figure that it can get rid of it? > >Regards, >Alexandre BiqueIf you want to drop symbol versioning with llvm-objcopy: * llvm-objcopy -R .gnu.version -R .gnu.version_r in.so out.so * However, llvm-objcopy zeroes out the section content so at runtime glibc ld.so will error "unsupported version 0 of Verneed record". Thus we need to drop the DT_VERNEED tag as well. To achieve this, we can rewrite the DT_VERNEED tag to DT_NULL (0), terminating the dynamic table, effectively deleting all DT_VER* tags. DT_RELACOUNT is also deleted. Fortunately its absence does not affect correctness. This requires binary patching ... 0x000000006ffffffb (FLAGS_1) Flags: NOW - 0x000000006ffffffe (VERNEED) 0x8ef0 - 0x000000006fffffff (VERNEEDNUM) 5 - 0x000000006ffffff0 (VERSYM) 0x89c0 - 0x000000006ffffff9 (RELACOUNT) 1536 0x0000000000000000 (NULL) 0x0 In the end, the solution is: r2 -wqc '/x feffff6f00000000 @ section..dynamic; w0 16 @ hit0_0' a.so; llvm-objcopy -R .gnu.version -R .gnu.version_r a.so>On Wed, Nov 11, 2020 at 2:25 PM Alexandre Bique ><bique.alexandre at gmail.com> wrote: >> >> This did the job: >> >> patchelf --clear-symbol-version log Podolski.64.so >> >> Alexandre Bique >> >> On Wed, Nov 11, 2020 at 2:19 PM Alexandre Bique >> <bique.alexandre at gmail.com> wrote: >> > >> > On Tue, Nov 10, 2020 at 7:01 PM Fāng-ruì Sòng <maskray at google.com> wrote: >> > > > How to find what is pulling libmvec? >> > > >> > > If you build a -DCMAKE_BUILD_TYPE=Debug lld, you can set a breakpoint >> > > on lld/ELF/MarkLive.cpp:114 (ss->getFile().isNeeded) and check which >> > > symbol makes >> > > libmvec.so needed. >> > >> > I'm afraid this is going to far for me :) >> > >> > > > On the other hand, I don't understand why I have to specify that I >> > > > want exp from glibc 2.2 or 2.29, I just want the exp symbol, and if >> > > > possible the latest version available on the system. >> > > > Because I don't have a high precision requirement so they would all be >> > > > fine. Is that possible? >> > > >> > > Symbol versioning is complex ;-) While the runtime behavior is >> > > documented by Linux Standard Base Core Specification, Generic Part, >> > > there is no good documentation on the linker behavior. >> > > LLD has evolved to the current state after contributors "observe GNU >> > > ld behaviors and patch LLD when needed" :( >> > >> > I think there might be some discussions needed with >> > glibc/coreutils/gcc/llvm because it is way too complicated. >> > I believe that targeting an older glibc version should not be more >> > complicated than >> > -DGLIBC_COMPAT_LEVEL=0x020205 -Wl,compat=GLIBC_2.2.5 >> > >> > On OSX there is a target platform command line option, and I believe >> > it is the right approach, yet I can't really comment on it. >> > >> > > For an undefined symbol, there is only this form: sym at ver >> > > For a defined symbol, both sym at ver (non-default; sometimes called >> > > hidden) and sym@@ver (default) exist. >> > > >> > > For a defined symbol named sym at ver, it only resolves an undefined >> > > symbol sym at ver. >> > > For a defined symbol named sym@@ver, it resolves both undefined >> > > sym at ver and undefined sym. >> > > >> > > If you have glibc 2.29, there are exp at GLIBC_2.2.5 (non-default) and >> > > exp@@GLIBC_2.29 (default), your 'exp' undefined symbol >> > > will resolve to exp@@GLIBC_2.29 (default) and not work with older glibc. >> > > >> > > If you know that your translation unit does not define 'exp' (usual >> > > case), you can write >> > > >> > > .symver exp,exp at GLIBC_2.2.5 >> > > call exp >> > > or >> > > .symver exp,exp@@@GLIBC_2.2.5 >> > > call exp >> > > >> > > Then the undefined symbol 'exp' will be renamed to 'exp at GLIBC_2.2.5' >> > > and get bound to the old version at link time. It will thus work with >> > > old glibc. >> > >> > I have one .cpp file with: >> > >> > __asm__(".symver powf,powf at GLIBC_2.2.5"); >> > __asm__(".symver expf,expf at GLIBC_2.2.5"); >> > __asm__(".symver exp2f,exp2f at GLIBC_2.2.5"); >> > __asm__(".symver log2f,log2f at GLIBC_2.2.5"); >> > __asm__(".symver logf,logf at GLIBC_2.2.5"); >> > >> > __asm__(".symver log,log at GLIBC_2.2.5"); >> > __asm__(".symver log2,log2 at GLIBC_2.2.5"); >> > __asm__(".symver exp,exp at GLIBC_2.2.5"); >> > __asm__(".symver exp2,exp2 at GLIBC_2.2.5"); >> > __asm__(".symver pow,pow at GLIBC_2.2.5"); >> > >> > But it does not solve the issue. I've tried many variations. >> > >> > Do you know if it is possible after the link, to edit the shared >> > library and replace the dependency on GLIBC 2.29 to GLIBC 2.2.5? >> > >> > Regards, >> > Alexandre
On Wed, Nov 11, 2020 at 10:37 PM Fāng-ruì Sòng <maskray at google.com> wrote:> If you want to drop symbol versioning with llvm-objcopy: > > * llvm-objcopy -R .gnu.version -R .gnu.version_r in.so out.so > * However, llvm-objcopy zeroes out the section content so at runtime glibc ld.so will error > "unsupported version 0 of Verneed record". > Thus we need to drop the DT_VERNEED tag as well. > To achieve this, we can rewrite the DT_VERNEED tag to DT_NULL (0), terminating > the dynamic table, effectively deleting all DT_VER* tags. DT_RELACOUNT is also > deleted. Fortunately its absence does not affect correctness. > This requires binary patching > > ... > 0x000000006ffffffb (FLAGS_1) Flags: NOW > - 0x000000006ffffffe (VERNEED) 0x8ef0 > - 0x000000006fffffff (VERNEEDNUM) 5 > - 0x000000006ffffff0 (VERSYM) 0x89c0 > - 0x000000006ffffff9 (RELACOUNT) 1536 > 0x0000000000000000 (NULL) 0x0 > > > In the end, the solution is: > > r2 -wqc '/x feffff6f00000000 @ section..dynamic; w0 16 @ hit0_0' a.so; llvm-objcopy -R .gnu.version -R .gnu.version_r a.soThank you very much for your help. If I run: r2 -wqc '/x feffff6f00000000 @ section..dynamic; w0 16 @ hit0_0' a.so It results in a plugin that does not load anymore because: undefined symbol __at_exit, version Then if I continue with llvm-objcopy -R .gnu.version -R .gnu.version_r a.so; then the plugins can be dlopen() but when used something goes wrong and the plugin aborts. I'm not certain about continuing in this direction because it is getting very involved and if anything goes wrong difficult to support. Thank you so much Fāng-ruì Sòng for all your help and patience. Best regards, Alexandre BIQUE